All Articles

How To Solve Hard Problems

“Don’t worry if it doesn’t work right. If everything did, you’d be out of a job.” -Mosher’s Law of Software Engineering

One of the hardest things about managing is to empower junior engineers to tackle what can seem like unsolvable problems. I have been there myself, toiling away, changing line after line and re-running my code only to be infuriated after hours with no real progress. It never does anyone any good blurting out answers to problems. Simply telling an engineer how to solve a problem hurts both of us: it prevents the engineer from learning how to tackle a problem and it creates a dependency on me to always be there every time they feel stuck. At some point the training wheels need to come off.

To help you tackle hard problems I am providing a few of my favorite tactical, holistic and systematic methods that work for me. A caveat: the tactical approaches only apply to engineering problems (more specifically JavaScript problems). So for the non-engineers, skip down to the second section where we discuss the systematic and holistic approaches.


Tactical Advice

Setup your debugger early and use it often

Debuggers are powerful tools. For any new project my first step is to always setup a launch.json file in VSCode. This file allows us to define a debugger process to attach to the main process when launch the specified script. A simple example of this file for a node project looks like:

{
    "name": "Launch via npm",
    "type": "node",
    "request": "launch",
    "cwd": "${workspaceFolder}",
    "runtimeExecutable": "npm",
    "runtimeArgs": [
        "run-script", "debug"
    ],
    "port": 9229
}

With the accompanying NPM script looking like so:

  "scripts": {
    "debug": "node --nolazy --inspect-brk=9229 myProgram.js"
  },

It’s painful to watch an engineer spend hours inserting console.log() statements all over their code just to reason about the changing state that is causing a bug. One pass through with a debugger can pick up a bug that would take dozens of runs with only logging statements. Seriously, take the time to learn how to use your debugger.

Use structured logging

Logging is great when you need to uncover an issue in deployed code. However, it is near impossible to sort through thousands or millions of log lines that are just strings. For this reason, it is highly recommended you always use structured logging. Structured logs are formed in a standard way that allows for query ability and indexing. This can be in pipe delimited or JSON. Two of my personal favorites are:

Draw out your network diagrams

This is a great practice for any engineer. By spending time to understand the network request path of a system, you can get a better sense of potential fault points.

If you can’t draw the request path end to end then you do not understand the system.

Within your network diagrams you should have a firm understanding of the HTTP verbs being utilized and the potential status codes and meanings. These are fundamental things for every software engineer.


Systematic Approaches

The 5 Whys

This approach was made famous by Toyota and was initially used to find issues within their manufacturing process. It is a relatively simple yet powerful technique for determining the root cause of a problem by repeatedly asking “Why?” Each time you ask why, the answer provides the context for the next question. An example of this is as follows:

   The problem: I Got a speeding ticket
• Ask the question: “WHY?” – Late for work
• Ask the question: “WHY?” – Woke up late
• Ask the question: “WHY?” – Alarm didn’t work
• Ask the question: “WHY?” – Batteries were dead
• Ask the question: “WHY?” – Forgot to replace them

With the analysis complete we can now go and replace our alarm clock batteries and should not expect the cascading series of events to occur again. This technique is simple and can be used both in your personal and professional life effectively. It forces you to be sincere about the cause of issues and will not let you evade responsibility.

Root Cause Analysis

Root cause analysis a process in which issues are identified based on evaluating when the last normal process occurred. There are 4 steps to utilizing RCA to get to the core of an issue and solve your problem:

  1. Describe the problem clearly on paper.
  2. Determine a specific timeline from the normal situation to when the problem occured.
  3. Identify any causal factors that may be a part of this (e.g. timeouts, status codes, bad inputs).
  4. Graph the relationship between these causal factors to see which one sits at the heart of the problem.

You will notice step 4 is similar to the process of drawing out your network diagrams. Understanding the fundamental relationships between parts of a system can clarify breaking points.

Atomic Breakdown

This is a personal process I have utilized to handle challenging engineering problems. It involves following a functional mental model to separate system components into the smallest logical parts possible. In engineering this would take the form of breaking a larger piece of code into small functional units or classes that work in complete isolation. Once the problem is broken down into these parts we recursively test by adding in a new component on each pass. In effect, we build back to our original problem but in baby steps so that we can keep track of the current context. For engineers, this is effectively what unit testing and integration testing provide us.


Holistic Approaches

Take a walk

The Ancient Greek philosopher, Aristotle, was famous giving lectures while walking. There is good reason for this. Walking increases blood flow to the brain, can increase cognitive processes, increases BDNF, releases endorphins, lowers risk of depression, and directly strengthens the hippocampus(the part of the brain responsible for memory) .

If you are stuck on a problem, go for a brisk 20 minute walk.

Explain it to a friend

One of my favorite ways of getting over the wall of an intractable problem is explaining it to my wife. She is wildly intelligent but knows very little about software engineering. It forces me to think through how I explain things and often times uncovers the heart of the issue I am having.

Pick someone to speak to about your problems who will ask good questions and will provide candid feedback. By not only thinking through the problem out loud, you will have the benefit of a clean perspective to help look for solutions. I strongly recommend explaining the problem to an audience that is mostly unfamiliar with the situation. In engineering, it rarely helps as much to explain the problem to the engineer who sits right next to you as it would be to explain the problem to an engineer in another department.

Sleep on it

According to the CDC, 1 in 3 adults in the United States does not get enough sleep. Sleep is critical to memory formation and several other cognitive processes. If you are stuck on a problem and can’t seem to make any progress, get some extra sleep.

In the meantime, I strongly encourage reading Dr. Matthew Walker’s book Why We Sleep. It will both enlighten and terrify you into getting more shut eye.