1
Here’s a video showcasing the platform and demonstrating how to easily manage new data and changes using the RAG stack of your choice: https://www.youtube.com/watch?v=-OdWRxMX1iA
If you want to try release.ai out, we’re offering a sandbox account with limited free GPU cycles so you can play around and get a feel for Release.ai: https://release.ai. We suggest playing around with some of the RAG AI templates and adding custom workflows like in the demo video. The sandbox comes with 5 free compute hours on an Amazon g5.2xlarge instance (A10 with 24GB VRAM, 8vCPUs and 32GB). You will also get 16 GB and 4vCPUs for cpu workloads such as web servers. You will be able to run an inference engine plus things like an api server, etc.
After the sandbox expires, you can switch to our free plan, which requires a credit card and associating an AWS/GCP account with Release to manage the compute in your cloud account. The free account provides 100 free managed environment hours a month. If you never go over, you never pay us anything. If you do, our pricing is here: https://release.com/pricing.
For those that like to read more, here’s the deeper background.
It’s clear that open source AI and AI privacy are going to be big. Yes, many developers are going to choose SaaS offerings like OpenAI to build their AI applications, but as open source frameworks and models improve, we’re seeing a shift to open source running on cloud. Security and privacy is a top concern of companies leveraging these SaaS solutions, which forces them to look at running infrastructure themselves. That’s where we hope to come in: we’ve built Release.ai so all your data, models and infrastructure stay in your cloud account and open source frameworks are first class citizens.
Orchestration - Integrating AI applications into a software development workflow and orchestrating their lifecycle is a new and different challenge than traditional web application development. Release also makes it possible to manage and integrate your web and AI apps using a single application and methodology.
To make orchestrating AI applications easier, we built a workflow engine that can create the complex workflows that AI applications require. For example, you can automate the redeployment of an AI inference server easily when underlying data changes using webhooks and our workflow engine.
Cost and expertise - Managing and scaling the hardware required to run AI workloads is hard and can be incredibly expensive. Release.ai lets you manage GPU compute resources across multiple clouds with different instance/node groups for various jobs within a single admin interface. We use K8s under the covers to pull this off. With over 5 years of building and running K8s infrastructure our customers have told us this is how it should be done.
Getting started with AI frameworks is time consuming and requires some pretty in-depth expertise. We built out a library of AI templates (https://docs.release.com/release.ai/release.ai-templates) using our Application Template format (which is kind of a super docker-compose: https://docs.release.com/reference-documentation/application...) for common open source frameworks to make it easy to get started developing AI applications. Setting up and getting these frameworks running is a hassle, so we made it one click to launch and deploy.
We currently have over 20 templates including temples for RAG applications, fine tuning and useful tools like Juypter notebooks, Promptfoo, etc. We worked closely with Docker and Nvidia to support their frameworks: GenAI and Nvidia NEMO/Nims. We plan to launch community templates soon after launch. If you have suggestions for more templates we should support, please let us know in the comments.
We’re thrilled to share Release.ai with you and would love to get your feedback. We hope you’ll try it out, and please let us know what you think!
David, Erik and I have worked together for almost 20 years starting with Erik and I meeting each other at an Internship right out of college. We met David in about 2003 when we were all working for RLX Technologies, which was one of the original blade server companies. Our early days were focused on systems management problems before VM’s were widely used. Interestingly enough a lot of the systems problems we are facing today are similar to the problems faced back then, just abstracted a few levels. Staging environments were hard then and they are hard now.
Since then, we’ve all pretty much stuck together, including co-founding IMSafer (detection of inappropriate conversations online for parents) together in 2006. David did take a break from us along the way as he became one of the early engineers at Etsy where he got a new perspective on systems challenges while Erik and I founded CarWoo! (YCS09 - online car buying made easy). David eventually made his way to CarWoo! and rejoined Erik and I after 4 years at Etsy where he was responsible for their search infrastructure and was involved in their DevOps systems. CarWoo! eventually was acqui-hired by TrueCar in 2014. We stayed on as the technology leadership team there for 5 years and led a major replatforming project that solved environments and enabled developers to iterate quickly.
Throughout our careers in doing systems engineering, starting companies, and being in and around technology there has been a universal difficulty in building environments that represent production and actually aid in getting work done vs being a bottleneck. As we’ve explored this problem with early customers, we’re getting similar feedback that’s reinforced our excitement to solve this problem.
We’re hearing some common themes as we talk with customers. Teams with just one or few staging environments have a big bottleneck in their process and can’t get product delivered as quickly as they’d like. The drift between production and the few staging environments a team may have is a problem. A lack of environments causes stakeholders to be out of the loop until really late in the development cycle and rework costs are high.
Engineering leaders are telling us how expensive in time, money and distraction away from core company objectives it will be to focus on this effort. Many times this is the reason this project (building a more flexible staging environment solutions) sits in the backlog and teams are just living with suboptimal velocity. Leaders that have invested in building an in-house solution have told us how complicated building it has been and aren’t happy that they’ve had to dedicate resources to this versus solving customer problems directly.
It’s not all bad though, as companies who have actually built and are maintaining adequate environment infrastructure have a distinct advantage in speed of delivery of complex applications. They are moving faster and that speed is a distinct advantage over companies without this capability. However, the cost of building this infrastructure is generally prohibitively high unless you’ve raised a lot of money or your application is extremely simple.
We’ve solved staging environments at every company we’ve started or been a part of throughout our careers and it’s always been the key to enabling us to move fast. We’ve learned that if developers have their own staging environment automatically created on every Pull Request, they can move incredibly fast. They are free to experiment, they can share work-in-progress changes with stakeholders early and they aren’t waiting for resources to free up to get their work done. We’ve seen ideas come alive while they were being built which has allowed them to iterate faster with more feedback and less rework.
We decided to build Release after spending countless hours talking about our experiences in our careers and when we’ve been the happiest at work. We’ve seen how powerful technology teams can be when they have enabling platforms at their disposal and how hard it can be when they don’t. We’ve always taken pride in making developers happy and more efficient. Release is the perfect avenue to solve a really big problem AND do work we love, that makes us happy.
As we started thinking about building Release, we leaned into the technology we were already familiar with, Docker, docker-compose, AWS and used that as our starting point. We felt that adding Kubernetes to the equation gave us a way to create these environments in a generic way where almost any application would run. We’ve tackled some complex environments for our early customers with lots of services and the interdependencies between them, including cloud native dependencies. Our ability to tackle complex environments has given us hope that the right technologies have emerged that make this possible now.
As we started exploring the idea, we knew Docker and containers were the baseline and we liked the starting point that docker-compose offered for running applications and defining environments. If you have a docker-compose we can run environments for you. We take that docker-compose and compile it into an application/environment definition (release.yml) that we automatically generate. Think of this as an abstraction that sits in between docker-compose and Kubernetes that gives you flexibility to define environments and resources that meet your needs.
From the application definition in the release.yml and your docker-compose we automatically generate all of the Kubernetes yaml files to run your environments. As a customer, you don’t need to know anything about K8’s. Whenever you do a PR we automatically generate all that’s needed to deploy and run your environment in K8’s. We’re interested to hear how much access customers would want to the raw K8’s ecosystem. To date we’ve had the opinion that customers shouldn’t have to deal with it, but would love to hear HN’s opinions on this.
If you’d like to give it a shot, request access at https://www.releaseapp.io We’d love to have you! We’ve tried to keep the model simple and just charge a fee for the number environments created each month by our users.
We’d love to hear your feedback. We’d also love to hear about how companies are handling this problem today. Your feedback is incredibly important to us, we know the HN community has a really unique perspective and appreciate you reading our launch post and making it this far in our launch story!