Meet the Team: DataHub Engineering
June 03, 2021
A common refrain around our offices is “Data is the lifeblood of the financial ecosystem.” Our DataHub Engineering team, which has members in New York, Princeton, and London, provides a distributed platform for hosting datasets, complete with managed data stores, search, discovery, batch analytics, and real-time stream processing. The collection of services for cataloging, publishing, and consuming data is called Bloomberg Data Services (BBDS).
Let’s first meet team leader Gauri Ranganathan. She joined Bloomberg 24 years ago, right out of graduate school. She has watched the company grow and change throughout her career, from several different departmental points of view. She has worked on BBDS since its inception in 2013, and has been instrumental in growing the platform from one user to today, when more than a thousand engineers use it to power their applications with data.
Tell us about your role as head of DataHub Engineering and what the team is responsible for.
Our goal is to ensure that high-quality content is cataloged, standardized, discoverable, distributed, and accessible in one place. The BBDS platform offers low-latency, high-availability data stores, pipes, stream processing, real-time change data capture, as well as distribution and discoverability services.
We host diverse financial datasets. To support market data, like NASDAQ, Bloomberg has dedicated infrastructure called Ticker Plant. For reference data, such as company financials for 48,000+ companies worldwide, news acquired by crawling billions of websites, economic data, information about all corporate, government, and municipal bonds, corporate actions, funds, and ETFs, we have BBDS Data Pipe, which is based on Apache Kafka.
What are some of the unique technical challenges your team needs to tackle?
The goal of the BBDS platform is to ensure all datasets delivered to Bloomberg clients are defined, cataloged, standardized, discoverable, and accessible in one place. This helps reduce the cost of data distribution and helps growth businesses like Enterprise Data, Alternative Data, and Quant Analytics to succeed.
The team’s tech stack includes various open source distributed systems like Apache Kafka, Apache Spark, Apache HBase, MySQL, Apache Airflow, and Kubernetes. Supporting this wide range of technologies requires expertise in many areas. One of our main challenges is finding the right balance between investing in the stability and reliability of our systems and delivering features to our users.
Briefly tell us about your career path.
When I first joined Bloomberg, the training program for Engineering was located at 499 Park Avenue and it lasted for 3 months. Senior engineers would come by and proctor the classes. We learned so much from their experience and quickly absorbed the Bloomberg culture of moving fast and being bold and open.
After the training class, I joined Data Systems (now Data Technologies in Princeton) to support our Global Data department’s operations. I had the opportunity to work in Fixed Income (Corporates, Government, Preferred and Loans) and Equity Index to support data acquisition and distribution to Core teams like Fixed Income, Enterprise, and Trading Systems. I had the opportunity to connect and grow my network with several teams across Engineering, Global Data, and Product. This helped me understand our core businesses and my team’s impact.
When a new opportunity arose to work on the Bloomberg Real Estate vertical — an online service for commercial real estate owners and brokers to provide transparency in operations, finance, and tenant management — I took on the challenge and gained expertise in the web tech stack.
I also had the pleasure of working on Bloomberg Wealth, a personal wealth management online service for individual investors, where I led the development of wealth data acquisition architecture, client communication applications like message and chat, and user onboarding.
In 2013, I joined BBDS in Princeton to help with the buildout of a data pipeline for Global Data’s diverse datasets and haven’t looked back.
What’s your strategy for choosing team members, particularly in terms of diversity (of skills, aptitudes, attitudes, experience, cultural backgrounds, gender/ethnicity, and abilities)?
We value engineers who have strong problem-solving skills and think outside of the box. We have a mix of both junior and senior developers in the team, which helps everyone grow. We look for enthusiasm, drive, curiosity, humbleness, and a willingness to learn when choosing team members. The challenges our engineers face are how to build reliable, fault-tolerant systems that are efficient and scalable in order to address the varying needs of financial datasets. Talent is universal and we believe a diverse team helps us become a stronger team and build a better platform.
How do you inspire women (both inside and outside of Bloomberg) to pursue careers in tech?
I volunteer with Big Brothers Big Sisters and help middle school girls get engaged in STEM activities. I mentor engineers in leadership both within and outside my team. I have helped and encouraged women in my team to present at conferences like the Grace Hopper Celebration of Women in Computing (GHC) and look forward to doing more of this in the future.
What skills do you look for when hiring engineers for your team?
We look for coding fluency in at least one systems programming language (e.g., Java, Go, C++) and a solid understanding of data structures and algorithms. We focus on problem-solving, communication, and analytical skills. We value experience engineering scalable low-latency services and learning from making distributed systems resilient. We love to see ownership, quality, empathy, collaboration, and a bias for action.
What are some of the factors driving the rapid growth/expansion of your team?
Given the myriad use cases our platform supports, there are several areas of growth. DataHub Data Pipe enabled financial data that was previously locked in silos to be made available as streaming datasets. Streaming data, starting with one dataset, Bloomberg Dividend Forecasts, now supports thousands and is growing fast to help internal organizations like Data Technologies, Financial Analytics, Equities, News, Alerts, Portfolios, Trading Solutions, and Enterprise to share data in real-time. Often, large-scale batch operations are needed for these datasets. Another product we manage, DataHub Batch, is processing 16 billion records each week.
For financial applications that use the DataHub platform as an application back-end to simplify data management and distribution, we need the platform to manage low-latency sharded SQL data stores with high availability. We are growing the team to pioneer the use of Vitess to solve these challenges. Data exploration and collaboration on these datasets using Jupyter notebooks is another area of growth for DataHub.
Another significant area of growth is our integration with Bloomberg Query Language (BQL) to make all data at Bloomberg easily queryable to support large analytics workloads.
As we grow DataHub, we also grow as infrastructure engineers by investing heavily in reliability, scalability, efficiency, observability, and ease of use of our services. But, we grow the most by learning from one another, sharing, trusting, and being there for each other.
How do you foster culture on your team? How has that changed with everyone working from home during the pandemic?
Our team culture is collaborative and values team success over individual accomplishments. It’s an open culture, so we share our ideas and encourage everyone in the team to voice their opinions. We stay in touch via persistent chats, asynchronous communication, and video meetings. Weekly meetings give everyone in the team a chance to get to know what others are working on and to announce upcoming changes and significant releases.
We have always been a distributed team, with engineers in Princeton, New York, and London, so we were used to written communication and collaboration not just within the team, but also with our vendors and open source Kafka, Spark, Vitess, and MySQL communities in which we are actively engaged. We were able to adapt to remote working without losing productivity or needing to change the way our team works. But, we sorely miss in-person team gatherings, walks to Central Park, an evening of drinks, an afternoon of tea at Princeton University, and the many other impromptu things that make our work environment special. The pandemic has also been a challenge for onboarding new hires and getting them to meet others in the team, but we have organized fun virtual events, games, and lunches.
“The challenges our engineers face are how to build reliable, fault-tolerant systems that are efficient and scalable in order to address the varying needs of financial datasets.”
– Gauri Ranganathan
Luke Ahn is a software engineer in the DataHub Stream Functions (DSF) team. Like many junior Bloomberg engineers, he previously had little knowledge of the technology that he now works with every day. But, with an eagerness to grow, easy access to mentorship, and copious learning opportunities, Ahn was able to quickly become a proficient Kubernetes developer.
Tell us about what you’re working on now and what your biggest challenge is. What inspires you most about it?
My work helps our downstream applications seamlessly consume streaming data. It mainly involves building a highly resilient and highly available data integration platform with robust recovery processes. The biggest challenge is that software that works with a small sample size tends to require more attention when it is put into production, where the data processing loads are larger. Keeping a close eye on its performance at scale is one of our challenges and is key to our success.
How did you find out about Bloomberg? What made Bloomberg stand out for you? Why did you choose Bloomberg?
Before joining Bloomberg, I was a student at Cornell Tech. Bloomberg representatives visited the campus and talked about the work they do and the impact they are making. Working in both the technology and finance sectors was one of my career aspirations, so Bloomberg was the right place for me. Furthermore, I was thrilled to contribute to my team’s work propagating Bloomberg’s financial data to various parts of the world. One of the key takeaways I got from meeting Bloomberg representatives was that once you join the company, you are given various resources such as mentorship, months of training, and the opportunity to choose a team that matches your interests.
How did the training program at Bloomberg help get you up to speed on our systems?
Having gone through the new hire onboarding and training process, I can say that the feeling that the company is investing in individuals creates a positive bond and strong commitment to succeed.
There are a number of ways to gain knowledge at Bloomberg. One way is through studying, by reading books and documentation. Another way is through hands-on experience, which teaches you best practices. Plus, there are many willing engineers available to hold your hands and guide you as you learn.
Tell us about the mentorship you’re receiving at Bloomberg.
I am currently working with Kubernetes, a technology with which I had little to no knowledge prior to joining Bloomberg. Seeing how the company is utilizing the technology and the tangible benefits Kubernetes provides, I realized it was one of the key technologies in which I wanted to develop my expertise. I received support not only from the team, but also from the members across the department, as I learned the ins and out of the technology and its best practices. Bi-weekly knowledge-sharing sessions, as well as weekly office hours, are hosted to help propagate knowledge and build a culture of continuous learning. Team leads may also create tiger teams to get through the job together.
How does the collaborative environment at Bloomberg create opportunities to learn new skills and expand your expertise?
One of the things that impressed me most is that Bloomberg is indeed technology agnostic and that management is more than willing to explore new technologies and tools to get the mission done. You will find they invest a reasonable amount of time and resources in your ability to pick up new skills — from blocked out time to learn to the ability to attend relevant conferences, such as KubeCon and other CNCF-hosted events.
“One of the key takeaways I got from meeting Bloomberg representatives was that once you join the company, you are given various resources such as mentorship, months of training, and the opportunity to choose a team that matches your interests.”
– Luke Ahn
Shain Mathews is a Senior Software Engineer with the DataHub Stores team in Bloomberg’s Princeton office. After working in front-end and middleware development for his first decade at Bloomberg, he became hungry for a new kind of programming challenge and joined his current team.
Tell us about what you’re working on now and what your biggest challenge is. What inspires you most about it?
My current team focuses on the data stores/services and our HBase storage back-end. The team uses a complex storage mechanism to store data in MySQL databases, HBase clusters, and Apache Solr search servers. This hybrid approach makes it possible to provide a variety of features like the efficient storage and quick retrieval of stored datasets to applications, in addition to providing batch processing and search capabilities. Maintaining a hybrid distributed system also has its challenges, such as the need to ensure data integrity across databases, scale the system and make it highly available at the same time. A lot of effort is put into abstracting out the complexities of the underlying system from the end-users who interact with the platform via simple REST APIs.
It looks like you’ve worked on several Bloomberg teams throughout the years. What motivated your most recent moves? What surprised you most about them?
I started off as a junior engineer working on front-end applications used by financial analysts. Being a UI developer, you work closely with our product managers and clients. This gives you a different perspective and encourages you to think like a user rather than just as a programmer.
After a few years, I shifted my focus to middleware development. Though I could leverage my programming skills, I quickly realized that you need to have a broader skillset to be a successful middleware engineer. In addition to developing reliable high-performance software, you are also required to think like a systems engineer. You need to have a deeper understanding of things like how operating systems manage processes, threads, memory, etc. You need to look at the impact your application will have on the machine it runs on and how it would work side-by-side with other applications.
About four years ago, I got an offer to join the DataHub team. I was excited to join the team that was trying to solve data storage and data distribution challenges faced by other engineering teams across the company. This team presented a totally different set of challenges than I was used to, such as how to build and maintain a highly scalable, available, and reliable distributed system capable of storing and distributing massive amounts of data. I was amazed to see the wide array of technologies that were being used and the tools and automation that had been built to manage such a complex system.
How do you foster a collaborative inclusive environment at work?
Team leaders and managers communicate their expectations clearly and every member of the team gets to revisit and recalibrate these goals at least twice a year. They seek feedback and suggestions from the team when setting milestones for the upcoming quarters and years, rather than forcing a set of to-dos on the team. There are sessions where the entire team gets together and brainstorm ideas. The team reviews the existing system and identifies opportunities for improvement and every member’s opinions are counted during these exercises.
We also meet without the managers every month to ensure there is an open discussion. Any new ideas or change recommendations are debated and subsequently forwarded to the managers as necessary. Our team also tries to make it easy for the new members to onboard and contribute quickly by providing a dedicated mentor, having 1-1s with all members, conducting technology/domain intro sessions, and maintaining up-to-date team documentation.
“I was amazed to see the wide array of technologies that were being used and the tools and automation that had been built to manage such a complex system.”
– Shain Mathews
Ryan Leslie is a Senior Software Engineer in the DataHub Core team. He works on the pipeline platform that enables messaging across Bloomberg’s many systems.
What’s your biggest challenge? What inspires you most about it?
I find it challenging to maintain an infrastructure that’s both highly reliable and scalable. It’s a delicate balance that we need to be mindful of. I’m inspired by seeing how many internal application teams have successfully adopted our platform and the creative ways they continue to use it.
How do you keep things interesting after so many years at the same company?
At Bloomberg, technology evolves quickly and there’s never a shortage of new things to learn. It’s constantly rewarding to find creative solutions to new problems.
Another thing we often do as engineers is debug systems to try and explain unexpected behavior. This requires a different mindset from writing code, and it’s fun to come up with ideas about why something happens and figure out how best to prove it. The subsequent “Aha!” moment can be very satisfying.
Tell us about the importance of mentorship at Bloomberg.
Early in my career at Bloomberg, I was fortunate enough to have worked with some really bright and talented senior engineers. It’s important to have good role models that encourage you to take pride in your work and strive to do your best. These days, I try to pass these same lessons down to newer team members. I also continue to learn a lot about successful project management from my managers.
“At Bloomberg, technology evolves quickly and there’s never a shortage of new things to learn. It’s constantly rewarding to find creative solutions to new problems.”
– Ryan Lesli
Check out some open roles with our DataHub Engineering team