We are excited to introduce the AI Toolkit for Visual Studio Code, a powerful VS Code extension available today in the Visual Studio Marketplace. AI Toolkit enables developers to explore, try, fine-tune, and integrate state-of-the-art models from Azure AI Studio and HuggingFace into applications.
The AI Toolkit is an evolution of the Windows AI Studio extension that was released in November 2023. The AI Toolkit for Visual Studio Code is our response to user feedback on Windows AI Studio that they need a cross-platform developer experience that simplifies the process of experimenting with new models in their applications.
AI Toolkit for VS Code
Key AI Toolkit for Visual Studio Code Features
AI Toolkit Model Catalog
The AI Toolkit includes a catalog of curated, current models from Azure AI Studio that have been optimized to run locally, greatly simplifying the process for discovering and trying models in your development environment.
Currently, the catalog includes models that run great on Windows and Linux—both on CPU and GPU—and we will be adding MacOS-optimized models soon!
Model Playground for Local Experimentation
The AI Toolkit includes a robust model playground, allowing you to experiment with small language models locally or in the cloud. This feature empowers developers to explore and understand the capabilities of various models in a controlled environment.
Advanced Fine-Tuning Techniques
Stay ahead of the curve with support for the latest fine-tuning techniques, including:
- Parameter Efficient Fine Tuning (PEFT)
- Quantized Low Rank Adaptation (QLORA)
- Flash Attention 2
These techniques ensure that you can fine-tune models efficiently and effectively, optimizing performance for your specific use cases.
Evaluations and Insights
The AI Toolkit supports model and prompt evaluations, providing valuable insights into model performance. By integrating with Azure AI Studio and prompt flow SDKs, you can easily create custom evaluations to measure the accuracy, coherence, safety, and other metrics for your application scenario.
Deployment Capabilities
Deploy fine-tuned models seamlessly to Azure Container Apps (ACA), Azure Kubernetes Service (AKS), or Azure AI Studio. While the current release offers deployment capabilities for development environments, we are actively working on enhancing these features to support production deployments.
High-performance inferencing on Windows
As part of building models in Windows, we have collaborated across many teams at Microsoft to showcase how to run an efficient inference pipeline utilizing ONNX Runtime and DirectML to target a large set of GPUs in the market. By leveraging DirectML in the inference stack, developers can enjoy all the accelerators exposed on the Windows platform, including GPUs, NPUs, and CPUs.
As part of this update, the new Playground experience runs optimized models for Windows. This allows developers to test the capabilities of models within VS Code and start building fine-tuning projects for their specific needs.
How to Get Started
The AI Toolkit for Visual Studio Code is shipped as a Visual Studio Code extension, so you need to install Visual Studio Code first, then install AI Toolkit from the Visual Studio Marketplace. Follow this guide to get started!
Join us at Microsoft Build
Don't miss the Maximize joy, minimize toil with great developer experiences session at Build, where we will showcase the AI Toolkit's capabilities, including a demonstration of using Azure AI Studio for model evaluations and the deployment process for Azure Container Apps. These features are designed to streamline your AI development workflow, making it easier than ever to integrate sophisticated AI into your applications.
We are committed to continually enhancing the AI Toolkit for Visual Studio Code, and we look forward to seeing the innovative solutions you create with it. Download the extension today from the Visual Studio Marketplace and start exploring the future of AI development.
Stay tuned for more updates and exciting announcements at Build!