Build, test and share your pipelines - No code required!

💧 Instill VDP (Versatile Data Pipeline) is a source-available tool, designed to streamline your data processing pipelines from inception to completion. If your goal is to develop versatile AI features using Large Language Models (LLM), Generative AI, Vision, or Audio models, Instill VDP empowers you to:

• Effortlessly connect to your unstructured data
• Build pipelines to enable diverse AI functionalities in your applications
• Visually test pipelines with a single click, viewing output at each stage
• Easily share your pipelines to showcase your work

☁️ Instill Cloud offers a fully managed public cloud service, providing you with access to all the fantastic features of Instill VDP without the burden of infrastructure management.

• 🚀 Accelerate AI applications by building end-to-end AI-powered pipelines for unstructured data up to 10 times faster
• 🔌 Utilize pre-built connectors to access data from various sources, powerful AI models, and third-party tools
• 🌟 Benefit from a no-code drag-and-drop pipeline builder, enabling quick and customizable application development
• 🪢 Choose between real-time inference (SYNC) and on-demand workload (ASYNC) processing modes
• 🧁 Enjoy a scalable API-first microservice design, offering an excellent developer experience
• ⚡️ Leverage high-performing backends implemented in Go
• 📊 Gain visibility into pipeline performance through a detailed dashboard
• 🤠 Access no-/low-code interfaces, making VDP suitable for every AI and data practitioner

• macOS or Linux - VDP works on macOS or Linux, but does not support Windows yet.

• Docker and Docker Compose - VDP uses Docker Compose (specifically, Compose V2 and Compose specification) to run all services at local. Please install the latest stable Docker and Docker Compose before using VDP.

Note Code in the main branch tracks under-development progress towards the next release and may not work as expected. If you are looking for a stable alpha version, please use latest release.

Execute the following commands to start pre-built images with all the dependencies:

The stable release version

$ git clone -b v0.15.0-alpha https://github.com/instill-ai/vdp.git && cd vdp

# Launch all services
$ make all

The latest version for development

$ git clone https://github.com/instill-ai/vdp.git && cd vdp

# Launch all services
$ make latest PROFILE=all

🚀 That's it! Once all the services are up with health status, the UI is ready to go at http:https://localhost:3000!

To shut down all running services:

$ make down

Explore our documentation to discover all available deployment options.

Instill VDP stands as a crucial project within our Instill Core stack. It provides an open-source AI infrastructure tailored for unstructured data, enabling versatile AI application development. Within this ecosystem, delve into source-available projects that enable you to construct flexible AI pipelines, enhancing your data processing abilities and converting raw data into actionable insights.

• 💧 Instill VDP: Streamline the unstructured data journey for AI applications
• ⚗️ Instill Model: Transform your applications with AI models
• 🗿 Instill Base: Essential Services for VDP and Model

To dive into Instill Core and Instill Cloud, we provide:

• ✨ Console for non-developers, empowering them to dive into AI applications and process unstructured data without any coding.
• 🧰 CLI and SDKs for developers to seamlessly integrate with their existing data stack in minutes.
  • Instill CLI
  • Python SDK
  • TypeScript SDK
• 🙌 Join our Discord server to exchange ideas about unstructured data processing, AI, MLOps, and get support from the Instill AI team. We're here to support you every step of the way!

• 📔 Documentation - Check out the documentation website.
• 📘 API Reference - Access http:https://localhost:3001 after make all, or simply run make doc to access the OpenAPI documentation. The gRPC protocols in protobufs provide the single source of truth for the VDP APIs. The genuine protobuf documentation can be found in our Buf Scheme Registry (BSR).

We strongly believe in the power of community collaboration and deeply value your contributions. Head over to our Community repository, the central hub for discussing our open-source projects, raising issues, and sharing your brilliant ideas.

Please refer to the Contributing Guidelines for more details.

See the LICENSE file for licensing information.