Peter Lyons

Distributed tracing is a powerful observability tool, and yet, many organizations see little return for their investment. It's a familiar story: there's an initial period of enthusiasm, developers implement the tool, use it for a few weeks, and then stop using it. Meanwhile, costs keep creeping up. This excellent talk explains why that happens, and what you can do about it. Sampling can be as low as 0.01%! 🤯 https://lnkd.in/gJRdZPjY by Paige Cruz via USENIX Association SREcon24

13

4 Comments

Having a ton of fun writing an ORM for V2 of Case Kit. Typescript has such an incredible type system. I have autocompletion for column names, the ability to include relations, relations of relations, etc etc etc... all returning properly typed data (no classes in sight). It's been really satisfying seeing it coming together, and it's been great getting my hands dirty with some (relatively) complex generics. And to head off the obvious question… why aren’t I just using Prisma? Well there’s a couple of things I’m doing differently that I think are marginally better: there’s no code generation involved here, it’s just types (inspired by Drizzle); there’s no Rust binary you need to deploy, and it hews much closer to Postgres behaviour and syntax (the aim here is not to abstract away which database you’re using). But the real blocker to me using Prisma was that its extensions don’t support modifying queries of nested relations - a hard requirement for me due to the way I have multitenancy and soft deletes set up. It’s not shown in the video but the Case Kit ORM lets you write simple middleware that will be applied to every level of a nested query. Perfect for setting timestamps or adding clauses on tenant id or things like that.

22

This is a bittersweet post (more on that later) and long. When I joined The ASSISTments Foundation over 4 years ago, our platform was a hodgepodge of code and frameworks accumulated over decades. Ruby on Rails, GWT, JSP, Java, an API with no standard, and a database and data structures that had long since lost their ability to scale with the kinds of things we needed it to support. All running on a few physical servers at WPI. There was SO much work to do if we were ever going to have a foundation on which to truly innovate. Over the last few years, we’ve moved our entire infrastructure to the cloud, rebuilt the teacher experience and student experience in Vue.js from the ground up, and launched a brand new product for school admins and instructional leaders. But the last step was the biggest: we had to refactor all five of our products to use our new RESTful API (SDK 3.0), migrate terabytes of content to our new database structures and build an entirely new content management and authoring system to support everything. Plus, we wanted to include some of the first new features not possible on the old system: new item types and an alpha version of teacher authoring for our longtime ASSISTments 1.0 users. On Tuesday, after well over a year of the most complex refactoring you can imagine, the team delivered flawlessly. Even though this is not the Oscars (and I certainly am not “the winner” of this), I’m now going to go over my time limit and praise the amazing team responsible for this: First, Director of Engineering Ryan Emberling and Senior Product Manager Cindy Starks organized and managed the entire project through the many, many curve balls we were thrown and pivots we had to make, building the management systems on the fly as we figured out this herculean task, with help from Michael Weingarth owner of our district admin product. Lead Designer Kyle Scheuing, working with our Director of Content and now Product Owner Dawn Peterson designed an intuitive but amazingly powerful authoring environment that is legitimately worthy of a design award if such things were given for authoring environments. Our engineering leads, Christopher Donnelly, Joan Wong, and David Magid—along with Ryan—self-organized in true agile fashion to lead the rest of the engineering team through countless hours of analysis, architecture decisions, code review, and God-knows-what-else. And finally, our amazing engineering team: Danita Forbes-Sample, Manuel Gonsalves, Rownok Nowrose, Brian Rojas, Jaime Torres, Jose Velazquez did months and months of work rewriting vast swaths of our codebase. I am so proud of this team, and honored to be able to work with them. Their achievement has set the stage for innovation at ASSISTments for years to come. The bittersweet part of this is that I will not be with them for that journey. I will miss this team incredibly, and I hope our paths will cross again.

44

8 Comments

I have a new blog post that discusses our journey in speeding up Docker builds for a large rails monolith - https://lnkd.in/gT9HVnWv

20

Did you miss my final #RAGHack stream, "Evaluating your RAG chat app"? Catch the recording here: 👇 https://lnkd.in/gSd32wXm I covered: 🤔 What does it mean for a RAG answer to be high quality? 🔍 Which factors affect RAG answer quality, and which affect it the most? 💻 Using promptflow-evals SDK to run GPT evaluators in Python 📊 Using my ai-rag-chat-evaluator CLI tools to generate synthetic ground truth data, run batch evaluations, and compare metrics across runs 🤷🏻‍♀️ Strategies to make sure your app can say "I don't know" at the appropriate times Grab slides from https://lnkd.in/g-9xSQvV #rag #openai #python #evals

20

Julia Wu and I recently led a “Speedy Routes Initiative” at Cedar to tackle timeout issues and improve site reliability. It involved improvements to monitoring, uWSGI and nginx configurations, Django views, and AWS infrastructure, all via a coordinated cross-team project. Get the full story in our latest Cedar Tech Decode Blog post below! #growwithcedar

49

2 Comments

Let's 🎉 celebrate 🎉 the fact that AFLAC users have the ability to use passkeys to speed their logins, improving the user experience, and reducing their customer service costs. But I can't overlook this quote: "Passwords are still in place for users who are not ready to move to passkey, or as an alternate path if needed." In other words, users are still subject to the risks of shared secrets. The security needle hasn't been moved in a material way and phishing resistance hasn't been achieved. Progress is good, but let's not oversell the benefits of passkeys in deployments when we haven't removed the risk of shared secrets or weak account recovery mechanisms. What do you think?

32

10 Comments

During my time at LoopQA, I've found that an effective model for leveraging automation involves a clear division of responsibilities between development and QA teams. I'll caveat this by saying, I generally advocate for developers to refactor the automation they break, but the reality is that most organizations aren't willing to invest their time into it. Hence, having a dedicated QA team for E2E tests often proves more practical. Here's a typical and highly functional workflow: Local Development: -Developers work on their local environments. -They run unit tests and integration tests to ensure their code is functioning as expected. Dev/Test Environment: -Code is merged into a shared Dev/Test environment. -Unit/Integration Tests: These tests must pass before any merge can occur. If they fail, the merge fails. -E2E Sanity Tests: These are also run, but they can fail for both valid and invalid reasons. Developers need to be adept at identifying actual bugs versus new legitimate failures. -Automation teams step in to refactor failing E2E tests to ensure stability. A common problem here is when dev teams opt not to refactor failing tests, leading to false negatives and potential issues down the line. Stage/Pre-Prod Environment: -After passing all tests, the code moves to the Stage/Pre-Prod environment. -Any merge to this environment is contingent on passing E2E tests along with unit and integration tests. Production Environment -Finally, the tested and validated code is merged into the Production environment. Key Insights: -This model ensures that unit/integration tests act as crucial gatekeepers, providing immediate feedback to developers. -E2E tests, maintained by the QA team, ensure overall system validation and highlight integration issues early. -Collaboration and clear division of responsibilities between dev and QA teams maximize the value of automation, leading to robust and reliable code delivery. #SoftwareDevelopment #QA #Testing #DevOps #SoftwareQuality #Automation #ContinuousIntegration #ContinuousDeployment

46

9 Comments