2024 November Meeting - Building LightDB: Beyond Traditional Databases to a High-Performance, Type-Safe Solution

What if you could combine the best aspects of key-value stores, relational databases, and graph databases, yet overcome the limitations of each? This session introduces LightDB, a type-safe, high-performance database solution that evolved from a need for greater flexibility, speed, and developer control. Drawing on lessons learned while testing MongoDB, PostgreSQL, Neo4J, and ArangoDB, LightDB was designed to blend the fast retrieval of key-value stores, the structure of relational databases, and the interconnectedness of graph databases—yet moves beyond the constraints of linear traversal typical in graph databases.

I’ll share how LightDB’s architecture optimizes data traversal and storage without locking developers into a single model, allowing more versatile and complex relationships to be expressed naturally. Leveraging Scala’s type safety and macros, LightDB reduces processing times from days to hours, all while offering a clean, intuitive API that emphasizes developer freedom and high performance.

Key Takeaways:

• Understanding key-value stores, relational databases, and graph databases: strengths, weaknesses, and use cases.
• How LightDB combines these paradigms to offer a flexible, powerful approach to data handling.
• Scala’s type safety and macros in LightDB for clean, reliable code.
• Moving beyond graph databases' limitations: how LightDB enables complex data relationships without arbitrary linear constraints.
• This session will give Java developers a deep dive into LightDB’s architecture and philosophy, showing how it moves data handling to the next level by combining diverse database strengths into one coherent solution.

I’ve built a career on architecting scalable, innovative systems that push the boundaries of what's possible. As the CTO of OUTR Software, LLC, I design platforms that handle absurd workloads with precision—because pushing the limits is where the fun begins. My toolkit includes Java, Scala, Kotlin, Rust, and Dart, and I enjoy mentoring those bold enough to join me on this experimental journey. I’ve worked with companies like eBay and Lightbend, always blending creativity with solid technical expertise to craft solutions that just work—even when they shouldn’t.