This is a long story, but it has some interesting twists and turns, and I promise that the journey will be worth it.
At one point in my career, I was working on a six-month contract. The management at this place was slow in letting me know whether the contract would be renewed, and I wasn’t particularly happy there, so as the end date drew near, I phoned a few recruiters to see what positions were available.
It was easy to find IT jobs in those days, and within a day, three companies had expressed interest in speaking with me. One company in particular, a foreign bank, was particularly aggressive. They set up an interview right away, and they wanted me to give them an answer that same day. I told them that I would need to consider the offer over the weekend (BS, but always a good tactic). I contacted the other companies and stressed that I would need to make a decision quickly. Eventually, I accepted the position, because I couldn’t stall them any longer.
My main job there was writing software that supported the accounting system. The internal workings of the system were developed by an outside vendor. They managed the debits and credits, depreciation, tax calculations, etc. The software that I supported managed all of the data that flowed into and out of the vendor system.
Inputs to the system included account balances, loan payments, information on trades (stocks, bonds, and other instruments), interest rates for bond pricing, and currency rates (e.g. how many Japanese Yen would a US Dollar trade for that day?).
Outputs included operations reports (risk, profit and loss, etc.), feeds to compliance systems (money laundering, insider trading), and regulatory reports for government agencies. The government report had to be submitted on time every day. The feds have the power to revoke your charter and shut down your operations.
To summarize, a lot of information had to flow in and out of this system, and it had to work smoothly, accurately, and on time, day in and day out. I was responsible for making that happen.
So, why was this company eager to hire me? Well, they had some crude software in place to do all of these tasks, but it didn’t work very well. Catastrophic failures were frequent, and those failures often happened at two or three o’clock in the morning.
When a bunch of people get called in the middle of the night, they aren’t happy. This happened over and over again, and my job was to fix it.
Getting a call to fix a complex problem at three o’clock in the morning is a horrible experience. You’re in the deepest part of your sleep cycle. Your brain has to wake up before you can even think through the problem. Even if you figure out what happened and take the appropriate action to fix it, you’ll worry that you missed something. It’s a like a nightmare, except it’s really happening.
Over the course of a couple of months, I examined each of these failures. I figured out which parts of the software was breaking and replaced the, with new code that understood how to handle the issue and, if possible, to recover gracefully without human intervention.
It didn’t happen overnight, but the results were a big improvement. A computer system that was infamous for failing several times a week became stable to the point where it had about one failure per year.
My boss was very happy. While I was working on the nightmare accounting system, he asked me to design a new system for document sharing and retention.
The idea was straightforward. If you take a photo with your phone, you can view it right away, but no one else can see it until you upload it to a system with global access, like Instagram or Google drive.
The new system that we were designing did the same thing except not with photos.
It stored reports and documents. People in offices all over the world needed to see reports as soon as they were created. There was no such thing as “the Cloud” in those days, so we created our own mini cloud for our own purposes. I created the software to load four thousand reports into this mini cloud every day, and to make sure that none of them were missing.
The mini cloud worked well. It enabled us to share documents with users in offices around the globe, and it stored them permanently. If in the future somebody needed to find an old report, they could get it from the mini cloud.
One day, management approached me about adding one more file the mini cloud for long-term storage. It was a very large file; we would receive a copy of it once a month from the Bloomberg company.
Some background: financial companies rely heavily on Bloomberg for information. This information is distributed in a number of ways, but one of the primary sources is a product known as the Bloomberg Terminal. It’s basically a screen that shows a lot of numbers and graphs that change in real time.
Today, you can install software to display live feeds of financial information on your home computer, but when the Bloomberg terminal first became appeared, it was far more advanced than anything else that was available. It looked like something from a science fiction movie. To this day, it’s a very popular tool for traders and financial analysts. Walk onto any trading floor anywhere in the world, and some, if not most of the desks will have a Bloomberg terminal.
One feature of the Bloomberg terminal is that is has a message app built in. That might not seem like a big deal today, but imagine if you had WhatsApp twenty-five years ago. It would have seemed amazing.
Traders use the messaging app to communicate when they need information quickly. They ask each other for prices and rates as they consider which deals and trades to make.
But people use Bloomberg messages for non-business communications, as well.
“We’re going for drinks. Want to join us?”
“What time?”
“Around 7”
“Sounds good!”
The reason that I know this is because the big file that we received from Bloomberg every month was a log of these messages. Every message sent or received during that month on a Bloomberg terminal by any person in my firm was in the file that I received.
I know what you’re wondering. Did I read the messages? The answer is only when I had to (more on that later). Most of the time, I didn’t read anything in the file. I could have if I wanted to - the file was not encrypted - but to do so would have been an invasion of privacy, and I didn’t feel right about that.
Let’s fast-forward a couple of years to the day that I received a call from the legal department. A call from the legal department is never good news. They don’t call you to you inform you that you’re the lucky winner of free legal services for a year. They call you because something bad has happened, and you have information that they need.
The lawyers wanted to me to search the Bloomberg message archive for messages sent and received by a particular employee on a particular day. This person, a lady, had attended a conference call where sensitive, non-public information was discussed. It was supposed to be attended by invitees only, but the employee let someone from outside the firm join the call. The person from outside was a reporter. He was also her boyfriend.
To make a long story a bit shorter, the reporter published the information, management figured out that the lady was the leak, and she got fired. Now, she was suing the firm for wrongful termination. The legal department needed her Bloomberg messages to build their case.
There were several messages between her and the boyfriend. I retrieved them from the archive and sent them to the legal department, thinking that that would be the end of it.
Some weeks later, the lawyer called me back. The case was going to trial. Apparently, the Bloomberg messages must have been important to the case, because the lawyer wanted me to go to court and testify as to how these messages were handled and stored.
To tell you the truth, I think that it would have been thrilling to participate in something like that. Unfortunately, there was a conflict. The court date was right in the middle of a week that I had planned to take off for vacation. This wasn’t just any vacation. It was a once-in-a-lifetime trip to Europe with my father and my son. A lot of planning had gone into it. The flights and hotels had long since been booked. The itinerary was packed. It wasn’t something that I could easily have delayed by a week.
I explained this to the lawyer and asked whether they could postpone the trial date. He said that it wasn’t likely, and that the lady, who was clearly in the wrong, would probably win her lawsuit.
I felt bad, but I couldn’t change my plans. I would have regretted doing so for the rest of my life.
I’ll get out of bed at 3 o’clock in the morning to attend to an urgent matter if I have to, but I’m not going to reschedule a family vacation. I’m dedicated to my work, but I am infinitely more dedicated to the people who love me.