The Era Where No One Knows Again

Chapter 10

I heard similar words when smartphones first came out.

That the world would change.

At first, I thought it was a bit of an exaggeration.

Even if you could use the internet on a mobile phone, it was just a small screen anyway, I thought. I saw it as merely an inconvenient transfer of what you could do on a PC.

But within just a few years, so much had changed.

When creating web interfaces, you had to think separately about mobile, and the job title "app developer" naturally emerged. Companies suddenly started talking about mobile strategy, and the existing systems needed new APIs.

But I wasn't at the center of the new wave back then either.

Someone built apps, and someone designed the mobile service architecture.

I connected the existing systems.

When orders came through mobile, I passed them to the existing order system, made existing member information usable in the app, and ensured the payment system and points system ran smoothly on mobile as well.

Behind the new interface was the old system.

And that old system didn't change as easily as expected.

Once, a problem arose in the mobile order project.

Orders that worked fine in testing were sporadically failing in production.

The app showed that payment was complete, but the order wasn't created.

At first, everyone suspected the mobile side.

"Isn't the app sending the request twice?"

"Doesn't it retransmit if the network is interrupted?"

We checked the logs.

The app was calling it normally just once.

The server received the request too.

That's where it got strange.

It was waiting for a response from a specific external system right before saving the order.

Sometimes that response was late.

When a timeout occurred, the entire order transaction was cancelled.

The problem was that the payment had already been processed.

A meeting was held.

Mobile developer.

Order developer.

Payment manager.

Third-party vendor representative.

Everyone said their system was working fine.

It wasn't wrong either.

It was true when looking at just their own part.

I drew the flow on a whiteboard.

App.

API.

Payment.

Order.

External system.

I drew arrows one by one.

"Payment ends here."

I pointed to the next box with my finger.

"It's waiting for this before saving the order."

Someone said.

"Was it originally called before the order?"

"It was on PC."

"There was no problem on PC."

"But mobile has different network conditions."

"The external system is the same though."

I thought for a moment.

"The call volume might have changed."

The conference room went quiet.

Traffic increased after the mobile service launched.

On PC, users naturally took time moving between screens, but the process was faster in the app. The extent to which calls were concentrated at specific times also changed.

The old structure was causing problems with the new usage patterns.

Eventually, we changed the order flow.

After payment was complete, we would save the order first, and handle the external system processing separately.

It sounded simple to say.

It actually took considerable time to make the change.

Touching the existing logic caused problems elsewhere.

The old batch process was watching that status value, the admin screen was using the same value, and the settlement program was connected to it.

Change one thing and three others would follow.

Every time such work came up, I played a similar role.

"Can you figure out what impact changing this will have?"

"Can you check why the old logic is structured this way?"

"There's some strange data in operations, can you take a look?"

There were people who built new things,

and I was called in when the new clashed with the old.

At first, I didn't like that much.

I wanted to work on new things too.

I wanted to build something worth putting in a presentation.

Applied new technology.

Built new architecture.

Improved performance several fold.

I wanted my name in sentences like that.

But most of what I did was hard to explain.

Existing system impact analysis.

Operational issue response.

Legacy integration.

Data correction.

None of it looked particularly impressive.

It was similar when I changed jobs.

The interviewer asked.

"What was your main role in your recent projects?"

I had to give a long explanation.

There was an order system,

there was an external vendor,

mobile was added,

there was an existing payment method,

and problems arose in between.

After I finished explaining, the interviewer nodded.

"So you've done a lot of maintenance work."

Hearing that made me feel a bit deflated.

Maintenance.

It wasn't wrong.

But it felt like a word that diminished what I had done.

For a while, I avoided that word in my resume.

System improvement.

Service stabilization.

Integration development.

I chose words that looked a bit better.

But as time went on, my thinking changed.

More than projects building new systems,

keeping an already-running system from stopping was often harder.

When building anew, if something went wrong, you could rebuild it.

In production, you couldn't do that.

Right now, someone was placing an order, paying money, and using points.

A wrong fix meant immediate trouble.

Even modifying a single piece of data required many checks.

Whether changing just this value would be okay.

Whether it wasn't connected to other tables.

Whether tonight's batch wouldn't revert it back.

Whether it wouldn't affect settlement.

No one knew the whole picture exactly.

Those who had worked there longer each remembered part of it.

"There must have been a reason it was changed like that back then."

"What reason?"

"I can't remember."

Conversations like that came up often.

Then we would search through old emails, find tickets, and dig through the source code.

Sometimes there were no records at all.

Then we would read the code and guess at the reason.

Why query this value again here?

Why save it twice on purpose?

Why was this condition added?

Code that looked odd at first sometimes made sense after you experienced an operational incident.

"Ah, so that's why it was added."

That moment was a bit enjoyable.

I thought it was like archaeology.

The work of finding traces someone left long ago and inferring the situation at the time.

Of course, most of them weren't that romantic.

I was in a hurry.

A call came from operations.

"Orders are not being placed right now."

"Since when?"

"It seems like about ten minutes ago."

From that moment on, recovery was more important than the reason.

Looking at the logs.

Looking at the servers.

Checking the recent deployment.

Checking the DB status.

I checked the status of external systems.

Several teams sent messages simultaneously.

Has the cause been identified?

Is there an estimated recovery time?

Customer service inquiries are coming in.

I replied.

Checking.

I wrote that a lot.

Sometimes I really didn't know anything.

But I had to do something.

I deleted them one by one.

DB is normal.

Server is normal.

External API response delay.

No recent deployment.

Then I saw something strange.

There were several times more specific requests than usual.

I followed the cause.

One button on the promotion page was incorrectly linked.

Users were making multiple requests with a single click.

It wasn't a problem that could be solved by increasing the servers.

Blocking the problem,

Organizing pending requests,

I confirmed that it was being normalized.

After about an hour, a message came from the operations team.

Normal confirmation.

I leaned back in my chair.

People around me got up from their seats one by one.

Someone said.

"You've worked hard."

That was the end.

The next day, everything went back to normal.

There was no presentation, and nothing major to write in the performance report.

The service had just stopped and started again.

As this happened several times, I realized.

That I wasn't the best at creating new things.

And strangely enough, it wasn't as embarrassing as it used to be.

When a problem arose, I knew where to look.

I knew who to ask.

I learned to look elsewhere when the code alone didn't provide an answer.

When people brought problems saying "Something is strange," I could roughly smell what was wrong.

It was a difficult-to-explain ability.

It wasn't on any certificate,

And it was ambiguous to put on my resume like a framework name.

But it was useful in the field.

Maybe I wasn't destined to be a developer who creates something great.

Instead, I became a developer who finds what's wrong with broken things.