How GitHub, Meltano, Airbyte, and Atlassian manage to stay focused on bigger goals while still staying flexible and agile. “As a PM, you must plan for the near term milestones (more detailed) as well as for the long term strategy (more broad), and everything in between. Considered as a spectrum, these form a nearsighted roadmap. This will enable you to efficiently communicate both internally and externally how the team is planning to deliver on the product vision.” (from the GitLab Product Handbook) ““In preparing for battle I have always found that plans are useless, but planning is indispensable.” (Dwight D. Eisenhower) […]
The Business Value of data products Is often miscalculated. Learn these two rules to calculate it correctly WSJF and the problematic Value for data products. Image by the author. For me, a product manager, the weighted shortest job, or what is called the cost of delay changed my perspective on understanding value. That’s what we want to do as product managers, maximize value. And the essential ingredient in that formula is the business value of a task or job. For data products, for data-heavy products, machine learning solutions, business intelligence systems, in short everything that has data at its core, I […]
Getting Started How to avoid the merge hell, speed up delivery of business value, reduce defects, and live happily ever after in your data warehouse. Faster development, fewer defects on deployment to production with trunk-based development in data workflows. Image by the author. “We needed an extra day to merge the transformation branches together”, “Ah yeah but there was a bug once we finally got the data to production, so we had to redo some stuff for another 2 days”,… sound familiar? To me, it seems like data and analytics engineers are particularly prone to run into the “merge hell” or […]
Use 7 simple questions to find machine learning opportunities, even without any technical knowledge (Photo by Markus Spiske, Unsplash) Machine learning, AI, Data Science all carry lots of scary and complicated concepts like deep neural networks, cross-entropy, optimization…. Enough scary words to scare off any product manager but the really tech-savvy from even thinking about integrating machine learning into their products at all. But that, in turn, makes it hard for a company to get all the value out of their machine learning engineers if most product managers shy away from employing them. I like to use a dead simple […]
In 2006, the company LinkedIn launched a new feature called “People You May Know”. This “prompt” turned out to have 30% higher click through rates than any other prompt in use at LinkedIn. It created millions of additional views and connections. The team also went on to create a bunch of additional machine learning products and helped to foster a deeply data-driven culture at LinkedIn. In their journey LinkedIn uncovered two important principles. The first principle is well explained by DJ Patil: “After all, what is a social network if not a huge dataset of users with connections […]
Dashboards, Graphs, Reports, Spreadsheets, OLAP Cubes, or direct SQL Access?
Data Meshes are the hot & trending topic in data & analytics departments. Implemented at big companies like Zalando, and moved from the “Trial” to the “Assess” status of the ThoughtWorks Technology Radar, within just one year. Yet the results I’ve seen are not overly impressive. Quite a few articles raising concerns have appeared throughout the past year, and at least I have gotten quite a bit of question & confusion about the topic after publishing my first article about data meshes.
Opinion On Redshifts, Data Catalogs, Query Engines like Presto, and the troubles of machine learning engineers to get their data. Image by the author. The author, confused between lots of different data mesh architectures. Data Meshes are the hot & trending topic in data & analytics departments. Implemented at big companies like Zalando, and moved from the “Trial” to the “Assess” status of the ThoughtWorks Technology Radar, within just one year. Yet the results I’ve seen are not overly impressive. Quite a few articles raising concerns have appeared throughout the past year, and at least I have gotten quite a bit […]
Refactoring, Working effectively with Legacy Code, and Test-Driven Development for Data Guys. …on software engineering. Hi, I’m Sven. I think data will power every piece of our existence in the near future. I collect “Data Points” to help understand this near future. If you want to support this, please share it on Twitter, LinkedIn, or Facebook. Here are your weekly three data points: Refactoring, Working effectively with Legacy Code, and Test-Driven Development. Why three software engineering books for data guys? Because I believe every data team should be treated as an agile development team. 1 Refactoring by Martin Fowler Whenever I take […]
Hi, I’m Sven Balnojan. I think data will power every piece of our existence in the near future. I collect “Data Points” to help understand this near future. If you want to support this, please share it on Twitter, LinkedIn, or Facebook. Here are your weekly three data points: Simple Data Discovery, Modern Data Architectures & Data Meshes. 1 Whale, a Dead Simple Data Discovery Tool “What does this column mean? Where can I find the order data?” Questions that bug every data engineer, every machine learner, and every analyst every day. There are a lot of powerhouse tools to […]