Meet the Team: Storage Engineering

Data is at the heart of Bloomberg’s technologies, which produce and distribute some of the most critical and valuable data in global business. The Bloomberg Terminal comprises thousands of different applications and analytics built by Bloomberg’s 6,500+ engineers — all of which are fed by some of this data. Hundreds of thousands of financial decision makers around the globe rely on Bloomberg’s financial data every day. One of the most critical components in this complex tech stack is a team upon which many others depend: the Storage Engineering team. This small, but vital squad is tasked with ensuring that over 100 petabytes of data are properly stored, secured, and always accessible.

First, let’s meet Engineering Manager Matthew Leonard, who leads the Storage Engineering team in Bloomberg Engineering’s Technology Infrastructure department. Matthew worked on software for fighter planes, and now leverages his “slow is smooth, smooth is fast” mentality from the aerospace industry to direct Bloomberg’s foundational data storage infrastructure engineers.

What is the Storage Engineering team’s mission?
We’re responsible for designing, building, automating, maintaining, and productizing Bloomberg Engineering’s storage technologies. This includes our SAN and NFS infrastructure, our SEC-compliant data stores, the persistent storage behind our private virtual compute platforms like ESX and Bloomberg Cloud Compute (BCC), as well as our next-generation software-defined storage stacks like our internal private S3 object store, Bloomberg Cloud Store (BCS), and NVMe-over-TCP initiative.

What are some of the technical challenges your team faces?
If we make a mistake, it can affect many other teams and their applications. So everything we do has to be deliberate and thoughtful. “Move fast and break things” is not an option for us.

Tell us about your career path.
During my pre-Bloomberg days I worked in the medical, telecom, and aerospace industries. While working in Montreal, Canada developing avionics software for the CF-18 Hornet, I was approached by recruiters from Bloomberg. Seven years later, I haven’t looked back.

What’s your strategy for building a diverse team?
It doesn’t happen by accident, and it’s one of the biggest challenges leaders in the tech industry face. One of the strategies that we use when assessing potential new teammates is to hire for aptitude and attitude. Mastery of a particular programming language or storage domain expertise are not barriers to entry. If you’re motivated and willing to learn, we will teach you everything you need to know to be successful in our organization. This approach exposes us to a broader and more diverse set of candidates.

What skills do you look for when hiring engineers for your team?
We have a diverse tech stack, ranging from massively distributed C/C++ systems, to more modern full-stack web-based development using frameworks like Flask, Swagger and React. Python is a big part of our automation projects, and we also dabble in open source projects like Ceph, HAProxy, and Apache Airflow. That list doesn’t even begin to get into all the specialized storage appliances and fiber optic fabrics our teams manage. So, given the breadth of technology and depth of domain knowledge in our area, we like to hire for aptitude and attitude.

How does your work writing software for fighter planes and satellites impact the work you do today?
There are a lot of parallels between my past experience and our current challenges. As I mentioned before, moving quickly and breaking things is not an option in Technology Infrastructure, and there is a similar ethos in the aerospace industry. In aerospace, you’re taught to expect the unexpected, build redundancies into your systems, and always have contingency plans, as people’s lives are on the line. In Technology Infrastructure, lives might not be on the line, but livelihoods are, so we tackle our challenges in a similar pragmatic approach.

“…everything we do has to be deliberate and thoughtful. ‘Move fast and break things’ is not an option for us.”

– Matthew Leonard

Engineering Team Lead Joseph Mundackal manages the Storage Distributed team, which focuses on providing Ceph clusters for Bloomberg’s private cloud compute and storage, and, more broadly, the company’s software-defined storage (SDS) ecosystem.

Tell us about your career path.
I started working for a company that builds set-top boxes for satellite TV in Colorado right after college. I got my graduate degree, then moved to NYC and started at Bloomberg, first in the application space, where I focused on investment banking workflows. I later switched to help with a migration effort mainly focused on security level analytics for indices. Now I manage the Storage Distributed team.

You’ve worked on a couple of teams since joining Bloomberg. What motivated your most recent switch?
My previous teams at Bloomberg were application teams. I learned a lot about UX and finance, but after about five years, I was looking for something more tech focused. After talking to a few people in my Bloomberg network, I learned about an opening in the Storage Engineering group. Being curious about cloud technology and knowing nothing about it, I was eager to jump in, head first!

What’s the collaborative environment like at Bloomberg?
Bloomberg is not so big that you get lost, nor is it so small that you lack opportunities to learn and grow. Every team and person I have worked with has helped me expand my technical skills and soft skills. Everyone is willing to help and point you in the right direction. One thing I would point out is that you need to take the initiative and not miss out on the various opportunities available, like recruiting, hackathons, tech talks, various Diversity & Inclusion (D&I) initiatives, and volunteering opportunities. 

Jane Zhu is rounding out her first decade at Bloomberg on the Storage Distributed team, after spending time on the Equity and Trading System teams.

You’ve been with Bloomberg for a decade. What has kept you here?
I didn’t expect I would stay this long when I first joined Bloomberg, but it’s been a fun journey. Bloomberg has such a dynamic and inclusive environment. It’s been constantly evolving and growing, and I’ve been growing with it. We have unlimited opportunities here to learn and experience, in both finance and tech.

How have you managed your career over the past decade to keep things interesting and challenging?
Never being complacent about what we have now, always finding ways to improve our systems, constantly expanding my knowledge, and trying to bring new technologies to our existing systems. These are some of the driving forces behind my progress in Bloomberg.

What has your experience been as a woman in financial technology?
Things have changed a lot in recent years, and we have more women in Engineering. I’m impressed by Bloomberg’s efforts in recent years to improve the work environment and promote career development for engineers who are female. We have more opportunities for women to express themselves, develop their technical skills, and advance in their careers across the organization.

How do you encourage diversity and inclusion within your team?
I promote D&I by attending Bloomberg Women in Technology (BWIT) events, giving tech talks, maintaining connections, and sharing ideas with female colleagues. I also try to make our work visible by attending the Grace Hopper Celebration of Women in Computing conference.

Software Engineer Ajitesh Chandak wears many hats in his role on the Storage API team. As a software engineer, he writes code to build better tools and automation platforms. 

You interned at a number of notable technology companies. What made you decide to start your post-academic career with Bloomberg?
Bloomberg always had a prominent presence at career fairs at UCLA, attracting a large portion of the student body — not only because they had a strong reputation, but also had the best swag!

Bloomberg is an interesting merger of technology with the financial world. The company is an influential market player, providing critical data to many financial professionals. After being offered a position, I flew to New York as part of a “Discovery Day.” The overall vibe made me feel at home. I just knew this was the place for me.

How does Bloomberg prepare its junior engineers to advance in their careers?
One might assume that being a junior engineer at a company would mean that responsibilities would be reduced, or that there would not be a lot of ownership on the services the team provides, or perhaps the projects would be smaller in scale. However, management places a high level of trust even in junior members, and nudges us out of our comfort zone to get wider exposure. I have been fortunate to have the opportunity to mentor a summer intern on our team this year, which has helped me learn the role a leader plays in people’s lives.

Have you found mentors to guide you along your way?
Every person I have met at Bloomberg has been a mentor to me to some degree. When I started, I had absolutely no understanding of data storage, except that my laptop had an SSD. Going from zero to understanding different storage technologies — and how they are leveraged in an enterprise setting — would not have been possible without mentorship from my colleagues.

What career advice would you give your younger self before graduation?
Never shy away from asking questions. It might seem that if you ask too many questions, you might appear unintelligent. On the contrary, it exemplifies your hunger to learn and curiosity to understand the details. A question unasked is a loss for you.

“Going from zero to understanding different storage technologies — and how they are leveraged in an enterprise setting — would not have been possible without mentorship from my colleagues.”

– Ajitesh Chandak

When Jenny Gao received a call from a Bloomberg recruiter 16 years ago, she’d never heard of the company. But that fateful call defined her career, and her crucial role on the Network Storage team.

Tell us about your career path.
I started as a systems engineer and project manager in Shanghai, China for an IT company. Information Technology was a new thing back then. A business card with an email address was not easy to explain to the customers. I worked on web servers and email servers for telecommunications companies and universities.

Then I served as a senior engineer in a big outsourcing company in Toronto, Canada, where I developed my passion for storage technologies, and became very good at it. Fibre Channel and SAN fabrics were new technologies. I dug up all the PDF manuals I could find at that time, reading them from cover to cover, again and again, to connect all the dots. I became one of the first Certified Fabric Professionals. I became the go-to person for my team for technical support.

When I got a call from a recruiter introducing me to a storage engineer position at Bloomberg, I had to ask the recruiter to spell Bloomberg three times so that I could write it down and check the company out.

What has kept you engaged at Bloomberg for 16 years?
I have worked on various projects which require a lot of coordination efforts with different application teams. I learned that everything we do on the storage team, every change we implement, every solution we deploy, could have an effect on the upstream applications. We are the foundation. We are providing the data service, not just a storage facility. I wanted to be the key person that the team can lean on, and rise to the occasion when critical moments come. It gives me pride when a complicated problem is resolved.

How have you seen the storage technology stack and technical challenges evolve over the years?
While one part of our job is to maintain and support current infrastructure, the other important and fun part of our job is to evaluate new technology, and to look for ways to improve the application performance, because we are I/O focused.

The storage team contributes to storage technology innovation in our own unique way. Bloomberg has a mixed application environment with a bursty workload, which demands a fast response and low latency. It inspires us to explore deeper beyond daily route operation, to think outside of the box, to locate bugs, to share our ideas, and most importantly, to push the storage vendors to take our suggestions seriously and implement the fixes and enhancements that eventually benefit their customers as well.

Bloomberg is full of interesting dynamics and opportunities, and so is the Network Storage team, which involves less than 10 team members effectively managing about 80PB of data across different underlying storage technologies. That says a lot about the team and skillset each team member brings to work. Getting our hands dirty with emerging technology keeps us current or ahead of the storage industry. That’s where my interest and heart is. I welcome the challenges it brings.

Check out open roles with Bloomberg’s Storage Engineering team