This blog post concerns the problems that arise from having too much skill. So on the surface it's a thinly veiled and pretentious humblebrag about a first world problem. It doesn't want to be that, and we hope that addressing the issue with this little lampshading helps.

What's the situation?

So. There you sit, happily writing code for your cool new AI-pipeline. You've been building these sort of things for a few years now, and you've gathered experience from projects of various types and sizes. Some of them have gone well while others have offered you various 'learning experiences'. In fact you were recently complimented by your boss in the team meeting for sharing your knowledge and experience! It really seems to be going great.

You keep on hammering on your codebase, noting from the clock that you still have an hour before a meeting with your boss and some salespeople about a project your company might want to work on. That's gonna be a nice break from typing at least. It's also nice to feel like your opinion is valued. Though to be honest, in the past few months you've noticed that people have been getting more and more eager to value your opinions in various meetings. On an almost daily basis. You've even noticed that sometimes you need to start adding blockers to your calendar so that you can dedicate contiguous blocks of your time for actually doing the work instead of talking about the work.

But you're sure most of that will blow over. You're continuously learning more and more about programming, the thing you really love to do, and surely the company will have to keep one of their better coders in the coding business!

The problem

The problem that is starting to arise here is not that you wouldn't be good in programming and getting even better. The problem is that you've started to amass more and more metaknowledge from the projects that you've worked on. You haven't only learned about how to write code but also about the various ways of how projects can succeed or fail regardless of code quality. Especially if your field is something like AI where things are advancing very fast, any sort of experience in contemporary systems and their design is highly valuable. In fact, it can be much more valuable than your coding skills.

This is even more true if you happen to spend most of your time codings something like AI-pipelines. In my experience, in most AI pipelines, 95% of the code is doing one of the following:

• Providing an API.
• Preprosessing input.
• Invoking CUDA/C/API endpoint of some 'eavy metal AI core or data wrangler.
• Postprocessing results.

In each of these the difference between bad code and good code is big, but the difference between good code and great code is not. After we get to the point where the code is robust-ish, readable-ish, documented-ish and maintainable with a reasonable test coverage -ish, there is not much left to do but make the code faster.¹ But if you speed up such code by a factor of 10, you're just going to spend a little less time before waiting for the execution of the C-libraries you are invoking that take 95% of the processing time. Possibly with the extra cost that now the junior Data Scientist cannot understand what your code is doing.²

On the other hand, if you have a person that can help a project manager to only shoot themselves in the foot a few times instead of a dozen times, you can save everyone a lot of time and effort.

Why is this a problem?

Well, this is mostly a problem only if you're like me and really like coding. Give me a partially defined idea that needs to be solved and I'm happy to hack together a throw-away piece of code to explore what is possible and what we really need. Or give me hard specs of a difficult problem and I'll start pumping up some proper test-hardened industrial-grade code.³ But it's harder to get to do that if your other skills are more valuable for your employer.

Of course in a good company your bosses are also interested in your well-being and job satisfaction. They won't sacrifice all of your happiness to the altar of short-term gains by giving you only repetetive tasks that maximize profitability. But they do have a pressure to give you some of that as well. And that probably means less programming and more strategic meetings.

To be fair, strategic meetings can also be kinda cool. And when you get to sit in meetings where the company or your team decides what to do, you can get to do more interesting things. And it's not at all unlikely that doing such things more will start to make you better at them, which in turn will make it even more interesting and fun.

So what's the punchline here?

I don't think there is a punchline. I think I'm just coming to terms with understanding what 'experience' means and thinking out loud is my way to process.

It is pretty much impossible to get very good in a given thing without boosting up the related (meta)skills, and the personal valuation you have for your skills doesn't usually exactly match their value to your business/employer/field. In the more general sense this is why it's good to have a separation between your work and hobbies, but in this particular case the difference has less obvious solutions. But observing the phenomenon is probably a good first step.

As a final remark, note that many (if not most) cultures have a thing about reverence towards the elders, and old age can be a good proxy for experience. If you have to estimate how long a given task will take, a very good idea is to try and remember how long it (or a task similar to it) took the last time. The older you get the more things you tend to have to compare things on, which is a huge benefit here. The age-skill correspondence isn't complete, of course. Getting older can make you⁴ more cynical and more reluctant to change. Furthermore in technology you⁴ might be tempted to use the tools you've learned instead of keeping up with the new trends, many of which might just blow over.

So maybe there is a sort of solution after all - the increasing value will be offset by the increasing crankiness until retirement? I'll try remember to report about this in a few decades.