We are excited to announce that Custom Categories is coming soon to Azure AI Content Safety. This new feature enables you to create your own customized classifier based on your specific needs for content filtering and AI safety whether you want to detect sensitive content, moderate user-generated content, or comply with local regulations. Use Custom Categories to train and deploy your own custom content filter with ease and flexibility.
Feature Overview
The Azure AI Content Safety custom categories feature is powered by Azure AI Language, a service that provides advanced natural language processing capabilities for text analysis and generation. The custom categories feature is designed to provide a streamlined process for creating, training, and using custom content classification models.
Here's an overview of the underlying workflow:
Deploy your custom category when you need it
We are offering two deployment options for our customers:
- Custom Categories (Standard):
- The Standard option for deploying custom categories is aimed at providing a thorough and robust filtering mechanism. It requires a minimum of 50 lines of natural language examples to train the category. This depth of training material ensures that the custom filter is well-equipped to identify and moderate the specified types of content accurately.
- Deployment Timeframe: The Standard option is designed with a deployment window of within 24 hours, balancing speed with the need for a comprehensive understanding of the content to be filtered.
- Custom Categories (Rapid):
- The Rapid option caters to urgent content safety needs, allowing organizations to respond swiftly to emerging threats and incidents. It requires a definition and few natural language examples for deploying the text incident, or few example images for deploying the image incident. This reduced requirement facilitates quicker creation and deployment of custom filters.
- Deployment Timeframe: This option emphasizes speed, enabling the deployment of new custom filters around just an hour for text, and few minutes for image. It is particularly useful for addressing immediate and unforeseen content safety challenges.
Both options serve to empower organizations with the capability to protect their AI applications and users more effectively against a wide array of harmful content and security risks, offering a balance between responsiveness and thoroughness based on the specific needs and circumstances.
How to use this feature?
Step 1: Definition and Setup
By creating a custom category, you are telling the AI exactly which types of content you wish to detect and mitigate. You need to create a clear category name and a detailed definition that encapsulates the content's characteristics. The setup phase is crucial, as it lays the groundwork for the AI to understand your specific filtering needs.
Then, collect a balanced and small dataset with both positive and (optional) negative examples allows the AI to learn the nuances of the category. This data should be representative of the variety of content that the model will encounter in a real-world scenario.
Step 2: Model Training
Once you have your dataset ready, the Azure AI Content Safety service uses it to train a new model. During training, the AI analyzes the data, learning to distinguish between content that matches the custom category and content that does not. Built on top of the underlying technology of LLM-powered low-touch customization from Azure AI Language, we are tailoring the experience for Content Safety customer towards consistency and more focus on content moderation scenario.
Step 3: Model Inferencing
After training, you need to evaluate the model to ensure it meets your accuracy requirements. This is done by testing the model with new content that it hasn't seen before. The evaluation phase helps you identify any potential adjustments needed before deploying the model into a production environment.
Step 4: Iteration
In the upcoming release of custom categories studio experience, we will introduce a feature that allows users to modify their definition and training samples using suggestions generated by GPT.
Join our customers using Custom Categories
South Australia Department for Education
“The Custom Categories feature from Azure AI Content Safety is set to be a game-changer for the Department for Education in South Australia, and our pioneering AI chatbot, EdChat. This new feature allows us to tailor content moderation to our specific standards, ensuring a safer and more appropriate experience for users. It's a significant step towards prioritizing the safety and well-being of our students in the digital educational space.”
- Dan Hughes, Chief Information Officer, South Australia Department for Education
Learn more about how South Australia Department for Education is using Azure AI Content Safety
Stay tuned!
Thank you for your support as we continue to enhance our platform. We are excited for you to begin using custom categories. Stay tuned for more updates and announcements on our progress.
In the meantime, we encourage you to visit our Content Safety documentation or studio to explore the existing capabilities available to you. Custom categories is also coming soon to Azure AI Studio and Azure OpenAI Service.