Dunya Oguz

The reason I'm learning more about analytics engineering is because I see a lot of demand. Even though there are more and more 3rd-party tools, the needs of in-house development is still there. With 3 Data Scientists, the productivity may be 3x of 1 Data Scientist. But with proper tools, the productivity growth can be exponential. This episode of podcast explain the idea from a simple but frequently seen scenario of we data folks face almost every day. Highly recommend the podcast host by dbt Labs for all data folks and even the leaders in tech. https://lnkd.in/gnkGHyZV

24

AI & ML is where all the fuzz is if you work in a data team. But most data teams have only just scratched the surface. dbt surveyed thousands of data professionals in their '2024 State of Analytics Engineering Report'. They found that only a third of respondents currently manage data for AI model training today. But 55% expect to soon. In this post, I've looked into the AI & ML landscape if you work in a data team, including the importance of building on top of solid foundations → State of AI in data teams → AI and ML use cases  → Data quality in AI and ML systems → The five steps to reliable data https://lnkd.in/e-6JTHQm

21

3 Comments

The last two weeks, we've run out-of-home campaigns (digital billboards) around #SanFrancisco. It also happened to be Snowflake's and Databricks' summits. From Lenny's newsletter and personal experience (as Cody Morgan found)... while we did see some top of funnel traffic bump, many of the biggest interactions with sticking our toe in the OOH pool was here on social. For most B2B products, it's difficult to efficiently target eyeballs given how little of the population is a buyer (if the goal is conversion). For awareness, we felt like we'd get the best bang for the buck in the area around the #MosconeCenter for the two big data conferences. There was not really "congestion" pricing in place (it tends to be uniform costs and then better dates sell sooner). I had a number of people tell me at the conference, "I saw your ads." So, at least anecdotally, it was successful for awareness. But a real tip to anyone doing OOH, the billboard is the billboard and social campaign around it.

24

1 Comment

💪 Super excited to finally share the new Spark Performance Optimization capabilities on the definity platform! ✨ Spark data engineering teams and platform leaders -- ready to unlock performance insights and cost savings from day-1 ? ⚡ Proud of the definity team who's moving at lightning speed to deliver tangible value to our customers. Check out the article by Tom Bar-Yacov (link below) and reach out to us to learn more!

32

3 Comments

Since joining Databricks in 2016, managing cluster costs has been THE challenge to maximize the best analytics plattform. After all, why not reduce costs and add more Use Cases? BTW - that is rhetorical 😉 Jeff Chou and his team at Sync Computing have built THE service to accomplish this goal! For any current or prospective #customer, this #Data & #AI Summit session promises to have the most impact on your business.....

1

Regulatory compliance with EIOPA is non-negotiable—but it doesn’t have to be daunting. We’ve built end-to-end data lineage across your entire data stack to ensure your data remains complete, accurate, and fully compliant with regulatory standards. Metaphor provides deep visibility into your data’s journey so that you can meet the stringent requirements of EIOPA Solvency II and DORA without breaking a sweat. Ensure accuracy across platforms and maintain data visibility that satisfies regulators and stakeholders. #EIOPA #DORA #DataLineage #DataCompliance

36

2 Comments

My friend Jesse Anderson’s latest survey turned up some interesting (and concerning) results regarding data teams: - 25% aren't using any form of LLMs, playing it safe or missing out? - Only 50% of data teams have dedicated operations teams, which means that despite the hard work of data engineers, data initiatives can suffer from poor implementation, misunderstandings, and misalignment with business goals. Key roles like DataOps engineers, data governance specialists, and quality analysts help ensure that all the brilliant insights generated translate into strategic action and real business value. #TIL Gartner "predicted" that the Data Mesh concept would be obsolete before the "Plateau of Productivity" (Data Mesh is indeed a concept, a strategy, not a clear-cut solution)—food for thought. 👀

4

If your data can consistently be parsed, it's structured; otherwise, it's unstructured.  The schema should evolve without breaking existing deserializers.  Structured data isn't limited to relational formats—it can be in different formats, as long as it can be parsed consistently.  If data can't be parsed consistently, it's unstructured.  That's the essence. There's no such thing as semi-structured—keep it simple.

6

Who among you is paying a fortune for Snowflake? And who has a boss who asks complicated questions, like “Why?” and “Why so much?” and “Can’t we find cheaper?” There is cheaper, but not as lovely. Not as effective and not as powerful. As for “why”: to get a fantastic analytical stack but, more importantly, to make well-informed decisions thanks to your incredible warehouse native experimentation tool, it’s strangely rewarding to remember the last 25 years of seeing data practices improve and the last 15 years of seeing the modern data stack evolve, but it’s great to have intelligent data processing, quality checks, dynamic alerts, automated partitioning, integrated systems of metrics, all those incredibly powerful abstractions serve what is the cherry on the MDS layer cake: experimentation. The Snowflake Summit is a wonderful celebration of that decade-long effort to improve data things. There are going to be a lot of vendors, including many of my colleagues from Eppo (don’t worry, a few of us will hold the fort while they fight with the conference Wi-Fi). If you are anything like me, all those logos and pitches can be overwhelming. The way I think about it is to step back for a bit of perspective: It’s great to feel that when the businesses you use to buy your travels, food, clothes, home, find a partner, or make a decision, they make it knowing all the consequences with accuracy and consideration. That’s what it’s about. And I’m very excited to be part of it, and you should be too. And you should go to the Snowflake Summit and when you are there, say hi to all my colleagues.

4

I've spoken to hundreds of data leaders about how AI will impact the modern data stack. Here are a few predictions on what we might see in the next couple of years: 🎨 Personalization: Every dashboard and report will be tailored to the specific job of the end user. One version for multiple users — marketing will see a tailored dashboard, and sales something different. 📝 Narrative: Data storytelling will reach new heights as writing narratives become more accessible to users. 🏗️ Extraction: AI will simplify data extraction from various sources, such as turning web pages into data (scraping), automating API usage, and handling unstructured data (e.g., PDFs, text, images). 🤓 Natural data querying: Text-to-SQL translation will become more advanced, making data querying more intuitive. ⏳ Time to value: Implementing BI tools, which currently can take months, will become much quicker with AI. 👩‍💻 Enhanced roles for data analysts: Data analysts will shift focus to more sophisticated use cases, models, research, machine learning, and AI applications, rather than just pulling data and creating dashboards for the business user. 🤝 Collaboration: AI will facilitate better collaboration among teams by providing shared insights and seamless integration with collaboration tools. Big changes are coming! I believe we’ll see a new wave of tools making data exploration and visualization much more intuitive for business users.

22

1 Comment

Throughout my career building and scaling data teams across various companies, from tech giants to scrappy startups, I've learned one thing: your data team structure is always evolving. Here's what I would recommend: 1️⃣ Start with an embedded model, the first few data analyst hires should have a dedicated focus area and work directly with your functional teams (e.g. product, marketing, finance). It makes it easier to secure budget approval and allows you to demonstrate the direct impact of data on key business outcomes. 2️⃣ Once you have the core models built for each functional team, it's time to centralize your data engineering function. This allows your engineers to focus on optimizing pipelines, refining tools, ensuring data quality and consistency, and driving cost-saving initiatives across the board. 3️⃣ Once your data team scales to 20+ people, data governance becomes more challenging. This is the ideal time to introduce a centralized data team responsible for building reusable data models (like dim_customer, fact_transactions) and standardizing metrics definitions across the organization. This central team ensures consistency and efficiency as your data needs scale. 4️⃣ As tech companies mature, the data team often follows a matrix structure.  Data analysts, specializing in different business areas, are embedded within functional teams like marketing or product, reporting directly to the Head of Data and dotted line to the respective business owner. And, there is a centralized data team that handles infrastructure and core data models, supporting these vertical teams with standardized solutions and owning organization-level initiatives. Building a high-performing data team is an ongoing journey, not a destination. It requires flexibility, collaboration, and a willingness to adapt to the ever-changing data landscape. What are your experiences with structuring data teams? Let me know what has worked for you in the comments below! 👇 #datateams #dataleadership #teamstructure #5X

21

1 Comment

The Bounteous Analytics team has been receiving a ton of requests for tracking iframes, so my colleague Badi Jones and I created a guide to help track interactions within iframes. iframes are known to be cumbersome in all aspects of web development from automation to tracking. They continue to be widely used today as reworking legacy codebase requires financial investment and human resources that are not always available. Additionally, iframes provide a layer of security for websites using them. iframes are likely not going anywhere; we created this guide and a started container, so you do not miss out on data that can be capitalized on. https://lnkd.in/gGQRaiGK

31

2 Comments