Toil Reduction: How Small Automations Freed Up Our SRE/DevOps Team

The origin of the term Toil

Documented around the year 1300, the word toil has meanings such as:

[hard work] toile, “turmoil, violent struggle, battle” (now obsolete senses), from Anglo-French toil (13th century), from toiler “to stir, disturb, entangle, twist,” from Old French toeillier “to drag, soil” (12th century). It is generally said (Watkins, etc.) to come from Latin tudiculare, “to crush with a small hammer,” from tudicula “mill for grinding olives, instrument for crushing,” from Latin tudes, “hammer” (from Proto-Indo-European tud-, variant of (s)teu- “to push, strike, pound, beat”; see obtuse).

The sense of “hard work, exhausting effort, labor carried out with fatigue and pain” (1590) comes from the related verb (see toil (v.)) or is reinforced by the notion of a “struggle” with difficulties and obstacles.. Reference Link

Toil in the context of SRE

In SRE, toil was the term chosen by Google to define tasks that are:

• Manual

• Repetitive

• Automatable

• Without lasting value

• That scale proportionally with service growth

At Monokera, we have a Slack channel called DevOps Requests, created during the company’s first generation of SREs/DevOps. This channel is still very useful today, as most requests to the SRE team come through it. However, it also concentrates a large amount of toil.

Some of these requests are excellent candidates for automation, for example:

• Creating and providing a backup of a specific database

• Deploying an API in AWS API Gateway

• Running a curl command on an endpoint

• Executing Rails tasks in specific environments

In daily operations, the SRE team is often focused on more complex tasks that require significant time and concentration. Then, one of these small requests comes through DevOps Requests, interrupting their focus.

Eliminating Toil

To eliminate toil, we have successfully created small automations and made them available as jobs in Jenkins. In this way:

• We increase development team autonomy and agility: Developers can execute the task themselves whenever needed.

• We ensure security and control: Jobs are configured to run only what is necessary and with predefined parameters, without requiring developers to access any environment directly.

This increase in agility positively impacts the entire development cycle, while the SRE team can dedicate more time and focus to more complex and strategic tasks.