# Rebuilding the engineering organisation behind a global social product
Source: https://manaienko.com/experience/askfm/
Author: Artem Manaienko

I rebuilt the engineering team while the product stayed live and the business was shrinking. I led up to 30 people through team leads, with responsibility for delivery, technology, hiring and the engineering budget.

**Role:** Chief Technology Officer

**Period:** Oct 2016 - Nov 2021

**Scope:** Up to 30 people, through leads

By the numbers

**Up to 30** people led through functional leads

**60%+** lower monthly infrastructure spend, approximately

**~20%** uplift to advertising revenue from video, approximately

**5 years** as CTO, reporting to the Board

[ASKfm](https://ask.fm/)

I was Chief Technology Officer at ASKfm from October 2016 to November 2021. I rebuilt the engineering function and led up to 30 people through team leads. I owned technology direction, delivery, reliability, hiring and the engineering budget, and reported to the Board.

The product was already live. We had to keep it running while transferring engineering ownership from Latvia to Ukraine, rebuilding the team and reducing costs as the business declined.

## Building the leadership team

I needed leads who could take responsibility for their people and delivery. I made good choices and a bad hire that I had to correct.

## Three decisions about leads

03 cases

-   ### Trying the responsibility before the promotion

    **Case:** I had two candidates: the strongest architect, and a more proactive engineer who communicated well with the team.

    **My decision:** I gave the second candidate two engineers and a complex feature, then stepped back to see how he led the work.

    **Result:** He delegated, delivered and earned the team's acceptance. I appointed him Backend Lead.

-   ### Correcting a bad QA Lead hire

    **Case:** An external lead interviewed very well, but showed little ownership and started demotivating the team.

    **My decision:** I dismissed him and promoted an internal engineer who already had the team's respect.

    **Result:** She became a strong lead. I had put too much confidence in the interview.

-   ### Preparing someone to take over from me

    **Case:** An Android Lead needed experience making decisions beyond his own team.

    **My decision:** I involved him in product statistics, infrastructure, architecture and organisational decisions, and gradually delegated more. He became my informal deputy.

    **Result:** He was Head of Engineering before I left. He later became CEO after other leaders departed; my part was preparing him for broader engineering responsibility.

## Changing how we released work

Shoutout promoted engaging questions to a wider audience. We planned roughly two months for the MVP and spent about six as changing requirements forced parts of the architecture to be rebuilt. Product and engineering started blaming each other.

The feature did well: average concurrent users nearly doubled and monthly active users rose by around 30%, based on historical recollection. I still consider the delivery a failure. We should have re-estimated and agreed on scope and dates much earlier.

Release model

Before

### A date and a growing list

-   Choose a date and bundle work into the release.
-   Keep adding requirements and rebuilding around them.
-   Move the date when the work no longer fits.

After

### One main feature

-   Agree on the main feature and when it is ready.
-   Ship compatible fixes without making every one a blocker.
-   Re-estimate when the main feature changes.

I discussed the change with product and engineering leads, listened to objections and followed up privately with quieter engineers. The new rules made scope and date discussions clearer. Delays still happened, and I have no reliable before-and-after measure of delivery time.

When the main feature changed, the people accountable for delivery had to choose between scope, date and quality. Compatible fixes could ship alongside it; unrelated work could not keep expanding the mandatory scope.

I treated the backlog as a choice about where to spend engineering capacity. I started with the business outcome, cost of delay and operational risk, then checked dependencies and future costs. Architecture mattered where it changed cost, risk or options. Each decision needed a smallest useful change, a stop condition and a way to check the result.

## Reducing infrastructure cost

We changed the way services were deployed, how storage and media were delivered, and how much physical capacity the database needed. I worked with the infrastructure and backend leads on the trade-offs and sequencing while the teams kept supporting production.

Infrastructure and delivery

Before

### Long-lived hosts and reserved capacity

-   Services configured on individual hosts.
-   Storage and CDN capacity that was expensive to keep.
-   A database workload needing substantial RAM.

After

### Updated delivery and lower capacity needs

-   Docker-based delivery with Puppet and Jenkins.
-   Storage and media delivery matched to access patterns.
-   Database changes across MySQL, Redis and RocksDB.

**60%+** lower monthly infrastructure spend over the tenure, approximately

The reduction came from several initiatives together. It is an approximate historical figure, not an audited saving from one migration.

One proposal did not get funded. I wanted to archive inactive-account media and text, but the Board preferred near-term product work during cash pressure. I disagreed and understood the decision. I came back with a narrower RocksDB migration. It delivered about 20% lower RAM use against my roughly 40% forecast. We kept it because it still freed useful capacity.

## Supporting revenue

I owned the technical side of monetisation with a small engineering group: advertising SDKs and networks, waterfall logic, native, banner, interstitial and video formats, plus targeting support for direct sales. I did not own the whole commercial function.

Video advertising added around 20% to advertising revenue, by my recollection. That figure applies to advertising revenue, not total company revenue, and the original measurement is no longer available.

## Keeping the product safe and available

During one registration attack, each defence changed the attacker's behaviour. We correlated data to find the pattern and combined technical and behavioural controls. The wider work included Elasticsearch moderation and search, controlled exports for legitimate authority requests and image-hash safety collaboration. These were product and user-safety problems as well as technical ones.

We also had to preserve availability through team transfer, infrastructure consolidation and changes to deployment, databases and providers. I no longer quote an exact uptime percentage. The old core-journey metric was too coarse, and I cannot reconstruct the original calculation.

## My part in the work

| Responsibility | What I did |
| --- | --- |
| Decisions I owned | Technology direction, engineering organisation and budget, hiring, leads and release process |
| Personal work | Parts of the AWS media redesign, architecture and economic analysis, Board proposals |
| Work led through specialists | Backend, mobile, web, QA and infrastructure, through their leads |
| Decisions I influenced | Product priorities and Board allocation between engineering and product investment |

## What I would change

-   Try leadership responsibility before making a promotion or hire where possible.
-   Measure delivery before changing the process, and keep the dashboards and decision records.
-   Define the minimum useful result alongside a technical forecast.
-   Make the deputy role and delegated decisions explicit earlier.

## Technologies

| Layer | What we used |
| --- | --- |
| Infrastructure | AWS, Docker, Puppet |
| Delivery | Jenkins |
| Data and messaging | MySQL, Redis, RocksDB, Kafka |
| Search | Elasticsearch |
| Backend | Ruby on Rails |
| Mobile | Kotlin, Swift, Java |

What this chapter taught me

-   [**How I select and grow technical leaders** Choosing a lead, correcting my own hire and preparing a successor.](https://manaienko.com/how-i-work/selecting-technical-leaders/)
-   [**How I explain decisions people will not like** Shoutout, and the move to feature-centred releases.](https://manaienko.com/how-i-work/decisions-people-dislike/)
-   [**How I report results, including the misses** A missed RAM forecast and an uptime claim I stopped using.](https://manaienko.com/how-i-work/reporting-results/)
-   [**How I handle delivery when scope keeps growing** A two-month MVP that took six, and the release model that came out of it.](https://manaienko.com/how-i-work/delivery-when-scope-stops-being-honest/)
-   [**How I turn business priorities into a backlog** An archival programme the Board rejected, and the narrower ask that got funded.](https://manaienko.com/how-i-work/business-priorities-into-a-backlog/)