[Introducing Human-in-the-loop Evaluation for Agentic AI Observability\\ →](https://humansignal.com/blog/introducing-human-in-the-loop-evaluation-for-agentic-ai-observability/) # Blog - [Guide](https://humansignal.com/blog/categories/guide) - [Insights](https://humansignal.com/blog/categories/insights) - [GenAI](https://humansignal.com/blog/categories/genai) - [RAG](https://humansignal.com/blog/categories/rag) - [Integrations](https://humansignal.com/blog/categories/integrations) - [Documentation](https://humansignal.com/blog/categories/documentation) - [Company](https://humansignal.com/blog/categories/company) - [Product Updates](https://humansignal.com/blog/categories/product-updates) - [Case Studies](https://humansignal.com/blog/categories/case-studies) - [Benchmarks](https://humansignal.com/blog/categories/benchmarks) - [Tutorials](https://humansignal.com/blog/categories/tutorials) [Product Updates](https://humansignal.com/blog/categories/product-updates) [Introducing Human-in-the-loop Evaluation for Agentic AI Observability](https://humansignal.com/blog/introducing-human-in-the-loop-evaluation-for-agentic-ai-observability) Turn agent traces into structured human evaluations to uncover quality issues and drive improvements. Micaela Kaplan April 15, 2026 ## Most Popular - [GenAI](https://humansignal.com/blog/categories/genai) [Consensus Agreement for GenAI Evaluation](https://humansignal.com/blog/consensus-agreement-for-genai-evaluation) - [Product Updates](https://humansignal.com/blog/categories/product-updates) [A New Audio Transcription UI for Speed and Quality at Scale](https://humansignal.com/blog/building-a-better-ui-for-audio-transcription-at-scale) - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Manage Quality at Scale with Super-granular Agreement](https://humansignal.com/blog/manage-quality-at-scale-with-super-granular-agreement) - [Product Updates](https://humansignal.com/blog/categories/product-updates) [The Most Advanced PDF Annotation Interface Yet](https://humansignal.com/blog/new-pdf-ocr-document-ai-labeling-interface) - Turn agent traces into structured human evaluations to uncover quality issues and drive improvements. Micaela Kaplan April 15, 2026 - [Guide](https://humansignal.com/blog/categories/guide) [The 5 Metrics That Actually Move AI Into Production](https://humansignal.com/blog/the-5-metrics-that-actually-move-ai-into-production) A practical framework for turning AI metrics into clearer decisions about what to fix, ship, and scale. Sheree Zhang April 15, 2026 - [GenAI](https://humansignal.com/blog/categories/genai) [Consensus Agreement for GenAI Evaluation](https://humansignal.com/blog/consensus-agreement-for-genai-evaluation) Consensus agreement is key for evaluating generative AI. Learn how to use LLM-as-a-Jury and score output preferences. Micaela Kaplan April 2, 2026 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Manage Quality at Scale with Super-granular Agreement](https://humansignal.com/blog/manage-quality-at-scale-with-super-granular-agreement) Now Label Studio Enterprise has the most flexible and granular metrics to measure agreement and compare agreement across sannotators, models and ground truth. Alec Harris March 24, 2026 - [Guide](https://humansignal.com/blog/categories/guide) [Scaling AI Data Quality: Best Practices for Onboarding and Evaluating Annotators](https://humansignal.com/blog/scaling-ai-data-quality-best-practices-for-onboarding-and-evaluating-annotators) A practical workflow for onboarding and evaluating annotators with clear instructions, calibration, quality gates, reviewer feedback, and dashboards that keep labeling consistent as volume grows. Alec Harris March 12, 2026 - [Tutorials](https://humansignal.com/blog/categories/tutorials) [Use YOLO26 with Label Studio](https://humansignal.com/blog/use-yolo26-with-label-studio-for-fast-bounding-box-pre-annotations) Learn how to connect YOLO26 to a Label Studio project using the YOLO ML Backend so annotators can start from model predictions and focus on review and correction instead of drawing boxes from scratch. Micaela Kaplan February 12, 2026 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [The Most Advanced PDF Annotation Interface Yet](https://humansignal.com/blog/new-pdf-ocr-document-ai-labeling-interface) Explore the most advanced interface for long-form or complex document annotation at scale. Thoughtfully designed shortcuts and user experience will make your annotators faster, more precise, and happier. HumanSignal Team February 4, 2026 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [A New Audio Transcription UI for Speed and Quality at Scale](https://humansignal.com/blog/building-a-better-ui-for-audio-transcription-at-scale) A new, dedicated audio transcription UI built directly into Label Studio Enterprise, optimized for speed, precision, and long sessions HumanSignal Team January 23, 2026 - ## Never miss an update. Subscribe for news. Email\* UTM Campaign Subscribe - [Guide](https://humansignal.com/blog/categories/guide) [How to Scale Evaluation for RAG and Agent Workflows](https://humansignal.com/blog/how-to-scale-evaluation-for-rag-and-agent-workflows) A practical workflow for evaluating RAG and agent systems consistently through prompt, model, and workflow changes. HumanSignal Team January 23, 2026 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Building the New Human Evaluation Layer for AI and Agentic systems](https://humansignal.com/blog/new-evaluation-engine) Programmable. Embeddable. Multimodal. Meet the new evaluation engine powering custom interfaces for AI and agentic systems. Michael Malyuk January 14, 2026 - [Benchmarks](https://humansignal.com/blog/categories/benchmarks) [The Five Stages to Keeping Benchmarks Useful as Models Evolve](https://humansignal.com/blog/the-five-stages-to-keeping-benchmarks-useful-as-models-evolve) A practical maturity model for taking benchmarks from proof-of-concept to versioned, continuously evolving evaluation that keeps up with models, prompts, and agent workflows. HumanSignal Team January 7, 2026 - [Benchmarks](https://humansignal.com/blog/categories/benchmarks) [Building a Quality Estimation Benchmark: The impact of relying on AI judges](https://humansignal.com/blog/building-a-quality-estimation-benchmark-the-impact-of-relying-on-ai-judges) What happens when you let AI judge AI? A pioneer benchmark for quality estimation in machine translation. Sheree Zhang December 22, 2025 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Label Studio Wrapped 2025: Inside the Workflows Powering Today’s AI Teams](https://humansignal.com/blog/label-studio-wrapped-2025) See how Label Studio evolved in 2025 to support richer labeling, measurable quality, and deeper analytics for real-world AI systems. Label Studio Team December 11, 2025 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Track Team Performance with the new Label Studio Analytics](https://humansignal.com/blog/track-team-performance-with-the-new-label-studio-analytics) Track annotator and reviewer performance, spot outliers early, and guide fair compensation with real data. Sheree Zhang December 4, 2025 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Evaluate Multi-Modal Agents in Label Studio](https://humansignal.com/blog/evaluate-multi-modal-agents-in-label-studio) If you are building agentic systems, you need an intuitive interface for evaluation. That's why Label Studio now supports evaluating multi-modal agents across text, visuals, and application outputs. Alec Harris November 19, 2025 - [Company](https://humansignal.com/blog/categories/company) [HumanSignal acquires Erud AI to build the world’s frontier data lab](https://humansignal.com/blog/humansignal-acquires-erud-ai-to-build-the-world-s-frontier-data-lab) AI has outgrown the open web. HumanSignal Services is creating multimodal data from scratch for frontier AI labs, with Label Studio as the backbone. Michael Malyuk November 5, 2025 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Inline PDF Labeling in Label Studio Enterprise for OCR](https://humansignal.com/blog/inline-pdf-labeling-in-label-studio-enterprise-for-ocr) Label Studio Enterprise lets reviewers correct OCR directly on PDFs. Draw OcrLabels on the page, capture or fix text, and export structured records with page and coordinates. HumanSignal Team October 27, 2025 - [GenAI](https://humansignal.com/blog/categories/genai) [Generative AI Templates for Reinforcement Learning with Verifiable Rewards](https://humansignal.com/blog/generative-ai-templates-for-reinforcement-learning-with-verifiable-rewards) Label Studio turns human judgments into structured rewards with ready-made and custom templates, so you can collect preference data and fine-tune models such as a math tutor bot for better answers Label Studio Team October 9, 2025 - [Insights](https://humansignal.com/blog/categories/insights) [The Last Mile of AI: Agent Evaluation with Domain Experts](https://humansignal.com/blog/the-last-mile-of-ai-how-domain-experts-turn-traces-into-real-improvements) Observability shows what happened; domain experts explain why, and their structured reviews turn raw traces into targeted fixes that finally close the last mile. Nikolai Liubimov October 9, 2025 - [Integrations](https://humansignal.com/blog/categories/integrations) [Data Control, Without Compromise: Label Studio Enterprise + Databricks](https://humansignal.com/blog/data-control-without-compromise-label-studio-enterprise-databricks) Label Studio Enterprise now integrates with Databricks, letting teams annotate data in place while keeping governance, compliance, and audit controls intact. HumanSignal Team October 1, 2025 - [Insights](https://humansignal.com/blog/categories/insights) [All Model Training Is Adversarial Now](https://humansignal.com/blog/all-model-training-is-adversarial-now) Once deployed, every AI model becomes an attack surface, learn why adversarial training and continuous retraining are now essential for resilience. David Hayes September 17, 2025 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Introducing Chat: 4 Use Cases to Ship a High Quality Chatbot](https://humansignal.com/blog/introducing-chat-4-use-cases-to-ship-a-high-quality-chatbot) Discover four practical ways teams are using Chat in Label Studio to ship high-quality chatbots that thrive in real user interactions. Alec Harris September 17, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [How Legalbenchmarks.ai Built a Domain-Specific AI Benchmark](https://humansignal.com/blog/how-legalbenchmarks-ai-built-a-domain-specific-ai-benchmark) See how Legalbenchmarks.ai built a custom contract-drafting AI benchmark using an SME-first, human-in-the-loop framework with rubric-based scoring and LLM judges. Sheree Zhang September 15, 2025 - [Insights](https://humansignal.com/blog/categories/insights) [Why Most LLM Chatbots Never Make It to Production](https://humansignal.com/blog/why-most-llm-chatbots-never-make-it-to-production) Most conversational AI pilots fail not because of technical limitations, but because of lack of governance and trust. Fortunately there's a playbook for organizations that need to transform chatbot experiments into production-ready systems. Michael Malyuk September 12, 2025 - [Insights](https://humansignal.com/blog/categories/insights) [Build vs. Buy in the Age of Generative AI: Lessons from MIT’s 95% Failure Stat](https://humansignal.com/blog/build-vs-buy-in-the-age-of-generative-ai-lessons-from-mit-s-95-failure-stat) 95% of GenAI pilots fail not from bad models, but from bad strategy. The winners know when to buy copilots and when to build the feedback loops that make AI reliable. Michael Malyuk August 29, 2025 - [Documentation](https://humansignal.com/blog/categories/documentation) [Label Studio SDK 2.0.0: Your Migration Guide](https://humansignal.com/blog/label-studio-sdk-2-0-0-released-your-migration-guide) We’ve released Label Studio SDK 2.0.0, bringing expanded project settings, new Enterprise endpoints, and improved documentation. Here’s what’s new and how to upgrade. HumanSignal Team August 21, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [SAM2 vs YOLO for Bounding Box Labeling: Which Should You Use?](https://humansignal.com/blog/sam2-vs-yolo-for-bounding-box-labeling-which-should-you-use) A practical comparison of SAM2 and YOLO for bounding box labeling, with pros, cons, and how to use each model in Label Studio. Micaela Kaplan August 14, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [9 Criteria for Successful AI Projects](https://humansignal.com/blog/9-criteria-for-successful-ai-projects) Most AI projects fail from unclear goals and poor alignment. These nine criteria show how to build systems that scale, adapt, and deliver value. David Hayes August 12, 2025 - [Benchmarks](https://humansignal.com/blog/categories/benchmarks) [Evaluating the GPT-5 Series on Custom Benchmarks](https://humansignal.com/blog/evaluating-the-gpt-5-series-on-custom-benchmarks) GPT-5 is out now -- but how good is it, really? In this post, we'll show you how we created our own custom Benchmark to evaluate GPT-5. Sheree Zhang August 8, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [The Virtuous Cycle of Using Your Brain Alongside GenAI - A Case Study](https://humansignal.com/blog/the-virtuous-cycle-of-using-your-brain-alongside-genai-a-case-study) This case study shows how human oversight turns AI missteps into opportunities for better systems, documentation, and long-term reliability. Alec Harris July 30, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [Operationalizing Compliance with the EU AI Act’s High-Risk Requirements](https://humansignal.com/blog/operationalizing-compliance-with-the-eu-ai-act-s-high-risk-requirements) The EU AI Act brings enforceable demands for transparency, oversight, and auditability, this guide shows how AI teams can meet them through robust infrastructure. HumanSignal Team July 25, 2025 - [Case Studies](https://humansignal.com/blog/categories/case-studies) [How Mind Moves and HumanSignal Brought Trust to AI in Healthcare](https://humansignal.com/blog/how-mind-moves-and-humansignal-brought-trust-to-ai-in-healthcare) To evaluate a GenAI health assistant in a high-stakes setting, Mind Moves used Label Studio to build a scalable, human-in-the-loop pipeline rooted in trust and rigor. HumanSignal Team July 23, 2025 - [Benchmarks](https://humansignal.com/blog/categories/benchmarks) [How to Build AI Benchmarks that Evolve with your Models](https://humansignal.com/blog/how-to-build-ai-benchmarks-that-evolve-with-your-models) Designing effective LLM benchmarks means going beyond static tests, this guide walks through scoring methods, strategy evolution, and how to evaluate models as they scale. Micaela Kaplan July 21, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [What Is a Spectrogram? Understanding Sound Beyond the Waveform](https://humansignal.com/blog/what-is-a-spectrogram-understanding-sound-beyond-the-waveform) In this post, we explore how spectrograms work and why they’re essential for analyzing speech, music, and machine learning audio data. Micaela Kaplan July 17, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [Intro to Hidden Markov Models: When the State Is Hidden but the Signal Isn't](https://humansignal.com/blog/intro-to-hidden-markov-models) This blog introduces the core components and algorithms behind HMMs, from decoding hidden states to calculating sequence likelihoods. Micaela Kaplan July 8, 2025 - [Benchmarks](https://humansignal.com/blog/categories/benchmarks) [Why Benchmarks Matter for Evaluating LLMs (and Why Most Miss the Mark)](https://humansignal.com/blog/why-benchmarks-matter-for-evaluating-llms) Custom AI benchmarks play a crucial role in the success and scalability of AI systems by providing a standardized approach to running AI evaluations. Sheree Zhang July 8, 2025 - [Tutorials](https://humansignal.com/blog/categories/tutorials) [How to Label Multichannel Time Series Data for AI Models](https://humansignal.com/blog/how-to-label-multichannel-time-series-data) Learn how to visualize and annotate complex sensor or signal time series data across multiple channels within the Label Studio interface. Label Studio Team July 2, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [Data Labeling with GPT-4 in Label Studio: ML Backend Integration](https://humansignal.com/blog/data-labeling-with-gpt-4-in-label-studio-ml-backend-integration) This blog shows how to integrate GPT-4 with Label Studio to shift from manual labeling to faster label review. Use OpenAI’s API and the ML Backend to streamline text annotation and accelerate NLP workflows. Jimmy Whitaker July 2, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [How to Label Spectrograms for AI Models](https://humansignal.com/blog/how-to-label-spectrograms-for-ai-models) You can now label audio data for training or fine-tuning AI models using spectrograms. This blog outlines why you might use spectrograms and how to use them in Label Studio. Micaela Kaplan June 30, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [Markov Models: Chains to Choices](https://humansignal.com/blog/markov-models-chains-to-choices) This post breaks down Markov Chains and Markov Decision Processes with intuitive examples and a video walkthrough. Micaela Kaplan June 24, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [Your Data Engine Is the Moat – Here’s How to Own It.](https://humansignal.com/blog/your-data-engine-is-the-moat-here-s-how-to-own-it) Model weights can be rented, but your feedback loop is the moat. This blog explains how leading AI teams are reclaiming control over labeling, evaluation, and iteration, without sacrificing speed. Michael Malyuk June 23, 2025 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Label Studio’s Multi-View Video Plugin](https://humansignal.com/blog/label-studio-s-multi-view-video-plugin) Label Studio’s Multi-View Plugin gives you synced video players to compare frames before, during, and after, making video annotation faster and more accurate. HumanSignal Team June 20, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [The Rise of Real-World Robotics—and the Data Behind It](https://humansignal.com/blog/the-rise-of-real-world-robotics-and-the-data-behind-it) Robots need labeled data to navigate the real world. This blog explores why robotics annotation is uniquely complex and how teams can get it right. HumanSignal Team June 11, 2025 - [Documentation](https://humansignal.com/blog/categories/documentation) [Integrating Label Studio with Model Context Protocol (MCP)](https://humansignal.com/blog/integrating-label-studio-with-model-context-protocol-mcp) Large language models can reason and plan but they can’t act. Learn how to integrate Model Context Protocol (MCP) with Label Studio to automate and simplify your annotation workflows. Jimmy Whitaker June 2, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [Who Watches the Watchdogs? Evaluating LLM-as-a-Judge](https://humansignal.com/blog/who-watches-the-watchdogs-evaluating-llm-as-a-judge) Measure annotator agreement with Krippendorff’s alpha,even on incomplete data. This guide walks through a real example and explains what the results mean. Micaela Kaplan May 21, 2025 - [Tutorials](https://humansignal.com/blog/categories/tutorials) [How to Automatically Disallow Text in Annotations](https://humansignal.com/blog/how-to-automatically-disallow-text-in-annotations) Learn how you can use Plugins to create a simple content moderation interface that will alert annotators of disallowed text in task submissions. Micaela Kaplan May 20, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [How to Use Krippendorff’s Alpha to Measure Annotation Agreement](https://humansignal.com/blog/how-to-use-krippendorff-s-alpha-to-measure-annotation-agreement) Learn how to calculate Krippendorff’s alpha to measure annotator agreement, even with missing data. This guide walks through a real example step by step. Micaela Kaplan May 19, 2025 - [Benchmarks](https://humansignal.com/blog/categories/benchmarks) [Everybody Is (Unintentionally) Cheating](https://humansignal.com/blog/everybody-is-unintentionally-cheating) AI benchmarks are breaking under pressure. This blog explores four ways to rebuild trust, governance, transparency, better metrics, and centralized oversight. Nikolai Liubimov May 13, 2025 - [Insights](https://humansignal.com/blog/categories/insights) [How Model Context Protocol Connects LLMs to the Real World](https://humansignal.com/blog/how-model-context-protocol-connects-llms-to-the-real-world) The latest _In the Loop_ episode breaks down the Model Context Protocol (MCP)—a new standard for connecting LLMs to tools, data, and real-world actions. Learn how MCP enables practical, production-ready AI. Micaela Kaplan May 8, 2025 - [Tutorials](https://humansignal.com/blog/categories/tutorials) [How to Detect and Pause Spammy Annotations](https://humansignal.com/blog/how-to-detect-and-pause-spammy-annotations) Learn how to automatically mitigate suspicious annotations like spam, bots, and fraud with Plugins in Label Studio Enterprise. Micaela Kaplan May 8, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [3 Annotation Team Playbooks to Boost Label Quality and Speed](https://humansignal.com/blog/the-spectrum-of-annotators-how-to-match-your-labeling-workflow-to-your-team) Not every ML team looks the same and your labeling workflow shouldn’t either. In this guide, we break down three common annotation team setups and how to tailor your tools and processes to boost quality, speed, and scale. Alec Harris May 7, 2025 - [Tutorials](https://humansignal.com/blog/categories/tutorials) [How to Generate Synthetic Data with Prompts in Label Studio](https://humansignal.com/blog/how-to-generate-synthetic-data-with-prompts-in-label-studio) Kickstart LLM training or RAG pipelines with model-generated data. This guide shows how to create a project, write prompts, and review outputs step by step. Micaela Kaplan April 23, 2025 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Build Powerful Labeling Interfaces with Plugins](https://humansignal.com/blog/build-powerful-labeling-interfaces-with-plugins) Apply powerful customizations to your Label Studio projects with only a few clicks. Browse a library of popular Plugins for advanced use cases or write your own using JavaScript. Alec Harris April 17, 2025 - [RAG](https://humansignal.com/blog/categories/rag) [Why Human Review is Essential for Better RAG Systems](https://humansignal.com/blog/why-human-review-is-essential-for-better-rag-systems) Learn how to prioritize weak responses, streamline human-in-the-loop review, and use feedback to iteratively improve your retrieval, prompts, and models. Jimmy Whitaker April 11, 2025 - [Documentation](https://humansignal.com/blog/categories/documentation) [Building a RAG System with Label Studio](https://humansignal.com/blog/building-a-rag-system-with-label-studio) In this post, we share how we used Label Studio and its Prompts feature to break down tasks, synthesize QA pairs, and build a reliable RAG assistant. Max Tkachenko April 10, 2025 - [Tutorials](https://humansignal.com/blog/categories/tutorials) [How to Evaluate and Compare LLMs Using Prompts in Label Studio](https://humansignal.com/blog/how-to-evaluate-and-compare-llms-using-prompts-in-label-studio) Using LLMs to label data is fast—but is it accurate? This post shows two ways to evaluate model performance in Label Studio, with or without ground truth. Learn how to compare outputs, track accuracy and cost, and choose the right model for your workflow. Micaela Kaplan April 2, 2025 - [Tutorials](https://humansignal.com/blog/categories/tutorials) [Craft the Greatest GPTs 😸](https://humansignal.com/blog/craft-the-greatest-gpts) In this post, we walk through a creative use case for Label Studio’s AI Assistant and Prompts: generating rich, image-based narratives. Follow along to see how you can build a project that combines structured labeling with a touch of storytelling. Sheree Zhang April 1, 2025 - [Case Studies](https://humansignal.com/blog/categories/case-studies) [Tales from Our Community: Stop the Traffik](https://humansignal.com/blog/label-studios-role-in-stop-the-traffik-s-mission-to-end-human-trafficking) When Stop the Traffik lost years of labeled data, they needed a faster, smarter way to rebuild. With Label Studio, they transformed their approach—bringing structure to messy reports, integrating AI for pre-labeling, and uncovering trafficking patterns hidden in plain sight. HumanSignal Team March 27, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [Seven Ways Your RAG System Could be Failing and How to Fix Them](https://humansignal.com/blog/seven-ways-your-rag-system-could-be-failing-and-how-to-fix-them) This post explores seven common RAG failured, from missing top-ranked documents to incorrect formatting, and provides practical solutions to improve retrieval accuracy, ranking, and response quality. Micaela Kaplan March 19, 2025 - [Tutorials](https://humansignal.com/blog/categories/tutorials) [Testing SmolDocling with Label Studio: Evaluating OCR for Document Conversion](https://humansignal.com/blog/testing-smoldocling-with-label-studio-evaluating-ocr-for-document-conversion) SmolDocling is designed for end-to-end document conversion, extracting text, tables, and layout with high efficiency. But how well does it perform on real documents? In this post, we walk through testing SmolDocling’s OCR capabilities using Label Studio and a step-by-step notebook to help you evaluate its accuracy. Micaela Kaplan March 19, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [Evaluating Mistral OCR with Label Studio](https://humansignal.com/blog/evaluating-mistral-ocr-with-label-studio) Mistral OCR is setting a new standard for document understanding, but how well does it perform on your data? Using Label Studio, you can evaluate its accuracy, compare outputs, and fine-tune results. In this post, we walk through the process and share a hands-on handbook to get started. Micaela Kaplan March 14, 2025 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Pre-Label Image Captions with Label Studio Prompts](https://humansignal.com/blog/autolabel-image-captions-with-label-studio-prompts) With Label Studio's Prompts, you can now handle multiple image annotation types - from basic object classification to nuanced caption generation - in a unified workflow that maintains both efficiency and quality. Sheree Zhang March 11, 2025 - [RAG](https://humansignal.com/blog/categories/rag) [How Human Oversight Solves RAG’s Biggest Challenges for Business Success](https://humansignal.com/blog/how-human-oversight-solves-rag-s-biggest-challenges-for-business-success) RAG is transforming how businesses use AI, but without human oversight, its accuracy and reliability suffer. This blog explores the biggest challenges in RAG implementation and how human expertise improves data quality, retrieval relevance, and AI-driven decision-making. Nikolai Liubimov March 6, 2025 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Accelerate Labeling and Model Evaluation with Prompts—Now Generally Available](https://humansignal.com/blog/prompts-generally-available-in-label-studio) Accelerate pre-labeling and model evaluation with LLM-powered workflows, while ensuring quality and managing costs. The Prompts interface is now generally available in Label Studio SaaS with support for more data types, evaluation metrics, and models. Sheree Zhang March 5, 2025 - [RAG](https://humansignal.com/blog/categories/rag) [RAG: Fundamentals, Challenges, and Advanced Techniques](https://humansignal.com/blog/rag-fundamentals-challenges-and-advanced-techniques) Explore the fundamentals of RAG, its advantages over fine-tuning, and the challenges of implementation. Jimmy Whitaker February 26, 2025 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Bulk Labeling: How to Classify in Batches](https://humansignal.com/blog/bulk-labeling-how-to-classify-in-batches) Subject matter experts can save time labeling multiple tasks at once. Filter to label all matching tasks, or select from a grid view using Data Preview for closer visual inspection. Alec Harris February 12, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [Reinforcement Learning from Verifiable Rewards](https://humansignal.com/blog/reinforcement-learning-from-verifiable-rewards) Learn about Reinforcement Learning with Verifiable Rewards, one of the leading training strategies for injecting learning signals into LLMs, successfully employed by models such as DeepSeek R1 and Tülu 3. Nikolai Liubimov February 7, 2025 - [Insights](https://humansignal.com/blog/categories/insights) [Top 5 Most Successful Data Curation Strategies in DeepSeek](https://humansignal.com/blog/top-5-most-successful-data-curation-strategies-in-deepseek) Learn the data curation and human supervision techniques that we believe are crucial to DeepSeek’s success by examining technical reports from DeepSeek-R1, DeepSeek-V3, and its predecessors. Nikolai Liubimov January 30, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [Content Moderation Labeling with Label Studio](https://humansignal.com/blog/content-moderation-labeling-with-label-studio) Get started with content moderation labeling to classify potentially harmful data for training purposes, or evaluate the output of a model you already have in place. Micaela Kaplan January 28, 2025 - [Guide](https://humansignal.com/blog/categories/guide) [Evaluating Multi-Turn LLM Chat Conversations Using Label Studio](https://humansignal.com/blog/evaluating-multi-turn-llm-chat-conversations-using-label-studio) Evaluating multi-turn conversations with Label Studio helps identify performance gaps, refine responses, and improve user experience for a more effective virtual assistant. Jimmy Whitaker January 21, 2025 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [NLP Autolabeling with Label Studio Prompts](https://humansignal.com/blog/nlp-autolabeling-with-label-studio-prompts) Large Language Models (LLMs) are revolutionizing how we tackle data annotation, offering speed and efficiency that were previously unattainable. With Label Studio's **Prompts**, you can now handle multiple natural language processing (NLP) annotation types for a single data task in a unified Label Studio workflow. Sheree Zhang November 21, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [New! Improvements To Annotator Performance Dashboards](https://humansignal.com/blog/new-improvements-to-annotator-performance-dashboards) Annotator Performance Dashboards make it easer to manage labeling teams at scale and orchestrate all the internal and external resources needed to improve data quality, without sacrificing speed. Label Studio Team November 14, 2024 - [Insights](https://humansignal.com/blog/categories/insights) [Why Humans Still Matter in the World of GenAI](https://humansignal.com/blog/why-humans-still-matter-in-the-world-of-genai) While data annotation for LLMs may look and feel somewhat different than the data annotation of the past, it’s still a crucial step of the machine learning process. Micaela Kaplan November 13, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [Fine-Tuning OpenAI Models: A Guide with Wikipedia Data](https://humansignal.com/blog/fine-tuning-openai-models-a-guide-with-wikipedia-data) Go through the entire fine-tuning process on OpenAI’s platform—from preparing recently-posted Wikipedia data to estimating costs and deploying your fine-tuned model. Jimmy Whitaker October 29, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Accelerate Your Named Entity Recognition Tasks with LLM-Powered Pre-Annotations](https://humansignal.com/blog/accelerate-your-named-entity-recognition-tasks-with-llm-powered-pre-annotations) In this post, we’ll guide you through the process of using Prompts in Label Studio Enterprise to pre-annotate data for Named Entity Recognition (NER) tasks. Sheree Zhang October 17, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Better Training & Fine-Tuning Data With Label Studio Enterprise Quality Workflows](https://humansignal.com/blog/better-training-and-fine-tuning-data-with-label-studio-enterprise-quality-workflows) Learn the intricacies of data quality, strategies to build the data you need for training and fine-tuning ML/AI models, and how you can use Label Studio Enterprise to engineer your AI/ML success. Nate Kartchner October 10, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [More Specific Comments and Annotation Rejection Options](https://humansignal.com/blog/more-specific-comments-for-annotators-and-reviewers) We’re excited to release our latest improvements to Label Studio Enterprise’s quality workflow: the ability to attach comments to a specific piece within an annotation and more granular reviewer rejection options. HumanSignal Team October 10, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [Webhooks in Label Studio: When And How To Use Them](https://humansignal.com/blog/webhooks-in-label-studio-when-and-how-to-use-them) Learn when you should use webhooks vs. the API in Label Studio, and see examples of what you can do with webhooks. Caitlin Wheeless October 1, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [Object Detection With Ultralytics YOLOv8](https://humansignal.com/blog/object-detection-with-yolov8) Learn how to use Ultralytics YOLOv8 object detection model with Label Studio. HumanSignal Team September 26, 2024 - [Tutorials](https://humansignal.com/blog/categories/tutorials) [OpenAI Structured Outputs with Label Studio](https://humansignal.com/blog/openai-structured-outputs-with-label-studio) OpenAI’s new Structured Outputs feature allows you to ensure outputs conform to a defined JSON structure. In this blog, we’ll explore how to leverage this feature for various labeling tasks. Jimmy Whitaker September 24, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [New Ultralytics YOLOv8 Integration For Label Studio](https://humansignal.com/blog/all-new-yolov8-integration-for-label-studio) We’ve released a new version of the Ultralytics YOLO ML backend connector designed for YOLOv8 and YOLO11, which now supports advanced object detection, segmentation, classification, and video object tracking with Label Studio. Label Studio Team September 18, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [New! Video Frame Classification](https://humansignal.com/blog/video-frame-classification) We’re excited to announce a new feature that enhances Label Studio’s video labeling capabilities: video frame classification. Label Studio Team September 11, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [LLM Evaluations: Techniques, Challenges, and Best Practices](https://humansignal.com/blog/llm-evaluations-techniques-challenges-and-best-practices) Explore the topic of evaluation for LLMs, its importance, and how we should approach it. Learn how integrating systematic evaluations can help teams iteratively refine their models to meet real-world needs. Jimmy Whitaker August 29, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Improvements to HumanSignal Reviewer Workflow](https://humansignal.com/blog/improvements-to-humansignal-reviewer-workflow) We’ve updated the reviewer workflow to make it easier and more intuitive. Nate Kartchner August 27, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [How to Build and Evaluate RAG Applications with Label Studio, OpenAI & Ragas](https://humansignal.com/blog/how-to-build-and-evaluate-rag-applications-with-label-studio-openai-and-ragas) In this tutorial, we'll guide you through the process of setting up and using Label Studio in combination with Ragas (Retrieval-Augmented Generation Answer Scoring) and GPT-4 to build an optimized QA application. Jo Booth August 22, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [Key Considerations For Evaluating RAG-Based Systems](https://humansignal.com/blog/key-considerations-for-evaluating-rag-based-systems) Implementing RAG-based systems comes with challenges to be aware of, particularly in assessing the quality of generated responses. This article will walk you through some of those challenges. Jo Booth August 20, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Transform Your Labeling Workflows With Custom Scripts](https://humansignal.com/blog/transform-your-labeling-workflows-with-custom-scripts) Label Studio is already the most customizable labeling platform. We’re making it even more flexible with custom scripts. HumanSignal Team August 6, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [New LLM Evaluation Templates For Label Studio](https://humansignal.com/blog/new-llm-evaluation-templates-for-label-studio) Evaluate the output of LLMs and RAG pipelines with Label Studio using five new templates designed for human supervision of AI models. Micaela Kaplan July 31, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Accelerate Image and Video Labeling with Segment Anything 2](https://humansignal.com/blog/segment-anything-2) Connect Segment Anything 2 (SAM2) with Label Studio to accelerate image and video data labeling. Label Studio Team July 31, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [3 Ways To Automate Your Labeling With Label Studio](https://humansignal.com/blog/3-ways-to-automate-your-labeling-with-label-studio) Delve into three effective methods to automate your labeling using Label Studio, including examples and resources. Nate Kartchner July 18, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Automate Data Labeling with HumanSignal](https://humansignal.com/blog/automate-data-labeling-with-prompts) We just released exciting functionality that could transform the way your data science teams work: **fully-automated data labeling**! HumanSignal Team July 9, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Introducing LLM Evaluations and the HumanSignal Platform](https://humansignal.com/blog/introducing-evaluations-and-the-human-signal-platform) Introducing Evaluations, Prompts, and the new HumanSignal platform. These new features make it easier to build reliable generative AI for the enterprise. Read on to learn more! HumanSignal Team June 26, 2024 - [Tutorials](https://humansignal.com/blog/categories/tutorials) [Fine-Tuning Llama 3: Enhancing Accuracy in Medical Q&A With LLMs](https://humansignal.com/blog/fine-tuning-llama-3-enhancing-accuracy-in-medical-q-and-a-with-llms) In this article, we want to demonstrate a method of curating large datasets to reduce but not remove the cost for curating a high quality medical Q&A dataset in Label Studio and fine-tuning Llama 3 on this data. Jimmy Whitaker June 11, 2024 - [Tutorials](https://humansignal.com/blog/categories/tutorials) [Improving RAG Document Search Quality with Cohere Re-ranking](https://humansignal.com/blog/improving-rag-document-search-quality-with-cohere-re-ranking) This article is part of a longer series that will teach you how to develop and optimize a question answering (QA) system using Retrieval-Augmented Generation (RAG) architecture. In this tutorial, we are going to show you how to create a generator that builds responses based on those documents. Max Tkachenko June 6, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [LLM Evaluation: Comparing Four Methods to Automatically Detect Errors](https://humansignal.com/blog/llm-evaluation-comparing-four-methods-to-automatically-detect-errors) An ongoing challenge for Large Language Models (LLMs) is their tendency to hallucinate. In this article, we explore four methods to automatically detect these errors. Nikolai Liubimov May 30, 2024 - [Tutorials](https://humansignal.com/blog/categories/tutorials) [Optimizing RAG Pipelines with Label Studio](https://humansignal.com/blog/optimizing-rag-pipelines-with-label-studio) In this introduction to our tutorial series on optimizing RAG pipelines, we'll introduce an example question answering (QA) system leveraging a Retrieval-Augmented Generation (RAG) architecture and outline three methods for optimizing your RAG pipeline utilizing Label Studio. Max Tkachenko May 23, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [Do I Need to Build a Ground Truth Dataset?](https://humansignal.com/blog/do-i-need-to-build-a-ground-truth-dataset) The short answer is: it depends. Read on as we explore this topic further, uncovering the advantages and drawbacks of each approach to help you make an informed decision. Label Studio Team May 21, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [Enhancing Data Quality with Label Studio Automated Workflows](https://humansignal.com/blog/enhancing-data-quality-with-label-studio-automated-workflows) This post will take you through the intricacies of data quality, the strategies employed to build top-tier datasets, and how to use Label Studio Enterprise to engineer your AI/ML success. Label Studio Team May 15, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [New! Annotator Dashboard Helps Optimize Team Performance](https://humansignal.com/blog/new-annotator-dashboard-helps-optimize-team-performance) New reports & graphs inside Label Studio provide the data you need to accurately pay annotators, track performance, and allocate resources. HumanSignal Team May 14, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [What’s a Ground Truth Dataset?](https://humansignal.com/blog/what-s-a-ground-truth-dataset) Understanding the distinction between regular datasets and ground truth datasets is crucial for leveraging data effectively in machine learning and data analysis tasks. This article explores both concepts and digs deeper into the importance of ground truth datasets. Label Studio Team May 9, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [Fine-Tuning Generalist Models for Named Entity Recognition](https://humansignal.com/blog/fine-tuning-generalist-models-for-named-entity-recognition) Generalist models, like GLiNER, provide an excellent starting point for the tasks that they aim to solve. Fine-tuning these models offers us a way to improve their performance in the areas that we care about to solve business problems. Micaela Kaplan May 8, 2024 - [Insights](https://humansignal.com/blog/categories/insights) [We Need a Better Set of LLM Evaluations](https://humansignal.com/blog/we-need-a-better-set-of-llm-evaluations) Different models are naturally going to excel at different tasks (just like humans). For users — especially those building products — having visibility into those tradeoffs is going to be a critical part of the decision-making process. Label Studio Team April 29, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [Strategies for Evaluating LLMs](https://humansignal.com/blog/strategies-for-evaluating-llms) Sure, benchmarks are cool, but they don’t give you the feel or the intuition of how a model **_actually_** works. To get that, you’ve got to hack around with the model and throw real-world prompts at it — like you’d do in day-to-day tasks. Label Studio Team April 23, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [New: Securely & easily connect models to Label Studio Enterprise](https://humansignal.com/blog/new-securely-and-easily-connect-models-to-label-studio-enterprise) Harness Generative AI and ML models for pre-labeling, interactive labeling, and model evaluation. HumanSignal Team April 17, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Automating Quality Control With Label Studio](https://humansignal.com/blog/automating-quality-control-with-label-studio) Today we’re launching a new feature to get your most challenging tasks in front of additional annotators—automatically. HumanSignal Team April 3, 2024 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Create a High-Quality Dataset for Reinforcement Learning from Human Feedback](https://humansignal.com/blog/create-high-quality-rlhf-dataset) RLHF has enabled language models trained on a general corpus of text data to be aligned with complex human values. This article details how you can train a reward model for RLHF on your own data. Jimmy Whitaker February 12, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [5 Tips and Tricks for Label Studio’s API and SDK](https://humansignal.com/blog/5-tips-and-tricks-for-label-studio-s-api-and-sdk) These five tips for using Label Studio's API and SDK demonstrate these tools' powerful capabilities and flexibility for managing data labeling projects. From efficient project creation and task imports to advanced configurations and bulk data exports, Label Studio provides a comprehensive and streamlined approach suitable for beginners and advanced users. Jimmy Whitaker January 23, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [An Introduction to Retrieval-Augmented Generation (RAG)](https://humansignal.com/blog/an-introduction-to-retrieval-augmented-generation) Get a brief overview of RAG and how it relates to LLMs, learn when you might consider using RAG, and get a summary of some challenges based on current research you should be aware of should you choose to travel down this path. Nate Kartchner January 11, 2024 - [Guide](https://humansignal.com/blog/categories/guide) [Medical Data Labeling and Label Studio](https://humansignal.com/blog/medical-data-labeling-and-label-studio) From precise disease diagnoses to personalized treatment plans, accurately labeled data profoundly impacts healthcare. This guide explores the fundamentals of medical data labeling, its applications, and its evolution through AI. Label Studio Team December 18, 2023 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [New: Streamlined Filtering for User Management and Collapsible Cards for Ranker](https://humansignal.com/blog/streamlined-filtering-for-user-management-and-collapsible-cards-for-ranker) Introducing new filters for managing users at the organizational levels. Also new are collapsible cards for the ranker interface, making it easier to work with high volumes of answers and cards containing lots of text. Sean Lynch November 20, 2023 - [Guide](https://humansignal.com/blog/categories/guide) [10 Tips to Supercharge Your Data Labeling Efficiency](https://humansignal.com/blog/10-tips-to-supercharge-your-data-labeling-efficiency) When training Large Language Models and utilizing machine learning, the significance of precise and efficient data labeling cannot be overstated. Here are ten actionable tips to elevate your data labeling processes. Label Studio Team November 16, 2023 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [New: Quickly Load and Manage Large-Scale Taxonomies From External Sources](https://humansignal.com/blog/new-quickly-load-and-manage-large-scale-taxonomies-from-external-sources) The newest version of Label Studio Enterprise includes support for large-scale taxonomies from external sources. This allows teams to load, manage, and maintain well-defined taxonomies of hundreds of thousands of choices in less than a second. Sean Lynch November 14, 2023 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Introducing Label Distribution Charts for Label Groups and User Soft Delete](https://humansignal.com/blog/introducing-label-distribution-charts-for-label-groups-and-user-soft-delete) The newest version of Label Studio Enterprise includes two updates that provide granular visibility into outliers and reduce security risks from churned employees: label distribution donut charts for label groups and user soft delete. Sean Lynch November 7, 2023 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Our Vision for the Future of Reliable Labeling Agents](https://humansignal.com/blog/introducing-adala) We're delighted to share our latest open source project with you! Meet Adala: a groundbreaking new framework for implementing agents specialized in advanced data processing, starting with data labeling and generation. Michael Malyuk October 25, 2023 - [Company](https://humansignal.com/blog/categories/company) [Making Sense of Data Labeling Automation](https://humansignal.com/blog/automated-data-labeling) _From active learning to autonomous agents, learn the use cases, strategies, and tradeoffs for automated data labeling._ Label Studio Team October 20, 2023 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Label Studio Enterprise Achieves HIPAA Certification](https://humansignal.com/blog/hipaa-certification) At HumanSignal, our top priority is the security and privacy of our customers' data. Today, we're proud to announce that we have achieved HIPAA compliance. Label Studio Team October 19, 2023 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Create Custom Labeling Interfaces Faster With Autocomplete](https://humansignal.com/blog/create-custom-labeling-interfaces-faster-with-autocomplete) This month, we've released an update that will streamline project setup. Labeling Configuration Autocomplete eliminates the need to code when creating custom labeling interfaces or modifying existing templates. Sean Lynch September 20, 2023 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Manage and Restrict Access to Your Data at a More Granular Level With Project-Based Roles](https://humansignal.com/blog/manage-and-restrict-access-to-your-data-at-a-more-granular-level-with-project-based-roles) We're excited to release **Project-Level Roles**. These provide more granular access to your data and simplify managing internal and third-party annotator permissions. Label Studio Team September 14, 2023 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Revolutionize Your Audio/Video Labeling With Contextual Scrolling](https://humansignal.com/blog/revolutionize-your-audio-video-labeling-with-contextual-scrolling) We are excited to share some new functionality that will enhance your data labeling experience with Label Studio - read on to learn more! Label Studio Team August 29, 2023 - [Guide](https://humansignal.com/blog/categories/guide) [Four Pillars of an Optimal Data Labeling Process](https://humansignal.com/blog/four-pillars-of-an-optimal-data-labeling-process) The realm of data labeling is undergoing significant transformations, reflecting the dynamic nature of the tech industry. Here are some of the most notable trends and their implications. Label Studio Team August 23, 2023 - [Guide](https://humansignal.com/blog/categories/guide) [Enhance Your Data Labeling Workflow With a Machine Learning Backend](https://humansignal.com/blog/enhance-your-data-labeling-workflow-with-a-machine-learning-backend) Integrating a machine learning (ML) backend into the data labeling process for a labeling platform can significantly enhance the efficiency and accuracy of the process. Label Studio Team August 3, 2023 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Increase Labeling Efficiency With New Enterprise Dashboards](https://humansignal.com/blog/increase-labeling-efficiency-with-new-enterprise-dashboards) With the introduction of Project Performance Dashboards, we're making it easier than ever to track and optimize your data labeling projects. Nate Kartchner July 11, 2023 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Introducing Ranker for Fine-Tuning LLMs, Generative AI Templates, UI Improvements](https://humansignal.com/blog/introducing-ranker-for-fine-tuning-llms-generative-ai-templates-ui-improvements) We're excited to showcase some new features we've added to Label Studio Enterprise specifically designed to help create datasets for fine-tuning Large Language Models (LLMs) like ChatGPT or LLaMA. Nate Kartchner June 26, 2023 - [Company](https://humansignal.com/blog/categories/company) [Betting Big on People: Heartex Evolves into HumanSignal](https://humansignal.com/blog/introducing-human-signal) In our four-year journey as Heartex, we've successfully built Label Studio, a top-notch data labeling platform used by tens of thousands of organizations. Today, we're taking a bold step forward as HumanSignal, harmonizing human insights and feedback with AI progression. Michael Malyuk June 20, 2023 - [Insights](https://humansignal.com/blog/categories/insights) [Why You Need a Scalable Data Labeling Process](https://humansignal.com/blog/why-you-need-a-scalable-data-labeling-process) Learn how building a scalable data labeling process ensures that your ML models have enough accurately-labeled training data to be effective and efficient. Label Studio Team May 3, 2023 - [How to Build a Data Annotation Team](https://humansignal.com/blog/how-to-build-a-data-annotation-team) Explore the essential steps and guidelines to create a data annotation team that can actively contribute to creating reliable data models. Label Studio Team April 26, 2023 - [Case Studies](https://humansignal.com/blog/categories/case-studies) [Managing A Data Annotation Team: 5 Key Takeaways From Yext](https://humansignal.com/blog/managing-a-data-annotation-team-5-key-takeaways-from-yext) We recently held a webinar with Dr. Vera Dvorak, Machine Learning Operations Manager at Yext. We’ve pulled out a few key takeaways for you. Nate Kartchner February 9, 2023 - [Insights](https://humansignal.com/blog/categories/insights) [Report: Data science teams shift from model development to dataset development](https://humansignal.com/blog/label-studio-community-survey-trends-2022) As we wrap 2022, the Label Studio community survey reveals trends, investments and technology choices for data science teams in the year ahead. Lauren Sell December 27, 2022 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Commenting is now available in Label Studio](https://humansignal.com/blog/commenting-is-now-available-in-label-studio) We've added comments and notifications to Label Studio Enterprise. Label Studio Team November 21, 2022 - [Company](https://humansignal.com/blog/categories/company) [The Heartex Team Takes Algarve, Portugal](https://humansignal.com/blog/the-heartex-team-takes-algarve-portugal) The Heartex team celebrated growth and milestones hit in 2022 at our first team offsite—join us in 2023! Brandi Bergstrom October 21, 2022 - [Insights](https://humansignal.com/blog/categories/insights) [Guide: Building a Data Labeling Practice for Machine Learning and Data Science](https://humansignal.com/blog/guide-building-a-data-labeling-practice-for-machine-learning-and-data-science) Learn the four core pillars of data labeling — data, process, people, and technology — and how to build a successful data labeling practice. Label Studio Team October 19, 2022 - [Company](https://humansignal.com/blog/categories/company) [Label Studio Enterprise achieves SOC 2 Type II certification](https://humansignal.com/blog/label-studio-enterprise-achieves-soc-2-type-ii-certification) Enterprise customers can feel confident that their high standards for security and compliance are met while experiencing the convenience of SaaS. Nate Kartchner October 12, 2022 - [Insights](https://humansignal.com/blog/categories/insights) [The Future Is Intelligent Data Labeling](https://humansignal.com/blog/the-future-is-intelligent-data-labeling) Learn why going from manual data labeling to intelligent data labeling could be the key to saving time and cost. Nate Kartchner October 4, 2022 - [A Brief Introduction to Data Labeling](https://humansignal.com/blog/a-brief-introduction-to-data-labeling) Learn about data labeling from Heartex founder and CEO Michael Malyuk. Label Studio Team September 20, 2022 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Try Label Studio Enterprise with new example projects and free trial](https://humansignal.com/blog/example-data-labeling-layouts-and-free-trial) Get started with sample labeling projects for image annotation, natural language processing (NLP), audio annotation, and time series data with a free trial. Label Studio Team September 12, 2022 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Major update to our annotation UI](https://humansignal.com/blog/major-update-to-our-annotation-ui) The newest version of Label Studio Enterprise includes a major update to our annotations UI that makes the tool much more ergonomic, efficient, and ready to support larger, more complex tasks with dozens to hundreds of regions. Label Studio Team September 7, 2022 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [ICYMI: New features in Label Studio Enterprise](https://humansignal.com/blog/icymi-new-features-in-label-studio-enterprise) Improved user, workspace and role management with new SCIM integration, and UX improvements to speed up your team’s annotation and review workflows. Nikolai Liubimov July 19, 2022 - [Data Annotation 101: Common Data Types & Labeling Methods](https://humansignal.com/blog/data-annotation-101-a-basic-exploration-of-its-methods) An overview of the common ways to annotate data based on the type of data and business goals. Label Studio Team July 12, 2022 - [Insights](https://humansignal.com/blog/categories/insights) [Why Internal Data Labeling is the Right Choice for Data-Centric AI](https://humansignal.com/blog/why-internal-data-labeling-is-the-right-choice-for-data-centric-ai) Better ML/AI performance starts with accurate and consistent data, labeled by domain experts, accelerated by active learning. Mark Brandsetter July 5, 2022 - [Company](https://humansignal.com/blog/categories/company) [Growing the Heartex leadership team](https://humansignal.com/blog/growing-the-heartex-leadership-team) Joe Alfaro joins as VP of Engineering, Lauren Sell as VP of Marketing & Ecosystem, and Brandi Bergstrom as Head of Talent Michael Malyuk June 23, 2022 - [Company](https://humansignal.com/blog/categories/company) [Label Studio Enterprise is SOC 2 certified](https://humansignal.com/blog/label-studio-enterprise-is-soc-2-certified) Read about an important milestone in our ongoing commitment to operational excellence and data security. Label Studio Team June 1, 2022 - [Company](https://humansignal.com/blog/categories/company) [From 20,000 ft to $25M — how Heartex is driving the data-centric AI movement](https://humansignal.com/blog/from-20-000-ft-to-25m-how-heartex-is-driving-the-data-centric-ai-movement) We’ve hit a big milestone for the company—securing our next funding round of $25 million in funding led by Redpoint, with participation from all our existing investors, Unusual Ventures, Bow Capital, and Swift Ventures. Michael Malyuk May 18, 2022 - [Insights](https://humansignal.com/blog/categories/insights) [Data-Centric AI: What is it, and why does it matter?](https://humansignal.com/blog/data-centric-ai-what-is-it-and-why-does-it-matter) Data-centric AI is a rapidly growing, data-first approach to building AI systems using high-quality data from the start and continually enhancing the dataset to improve the model's performance. Data-centric AI is a modern approach to building AI where model accuracy is primarily dependent on data quality. Michael Malyuk March 16, 2022 - [Essential Data-Centric AI Tools for 2022](https://humansignal.com/blog/essential-data-centric-ai-tools-for-2022) Learn about the most popular technologies and tools data scientists and ML teams leverage to power data-centric machine learning. Michael Malyuk March 10, 2022 - [How to Build an Effective Data Labeling Strategy That Scales](https://humansignal.com/blog/how-to-build-an-effective-data-labeling-strategy-that-scales) Data labeling may seem simple, but it isn’t always easy to implement at scale. And getting it wrong will delay your entire model training process. Learn how to develop your labeling strategy for scale and accuracy. Michael Malyuk February 3, 2022 - [How to Manage a Data Science Team: 4 Things Every Leader Should Know](https://humansignal.com/blog/how-to-manage-a-data-science-team-4-things-every-leader-should-know) Building and managing a data science team has some interesting and unique challenges. How do you structure your team? What are the right roles? Nikolai Liubimov January 20, 2022 - [Company](https://humansignal.com/blog/categories/company) [2021, it’s a wrap!](https://humansignal.com/blog/2021-its-a-wrap) 2021 was a monumental year for Heartex and Label Studio. We innovated, built the largest data labeling community, and hired an amazing team. What's in store for 2022? Michael Malyuk January 5, 2022 - [Case Studies](https://humansignal.com/blog/categories/case-studies) [Bombora Powers Business with Machine Learning, Data Science, & Heartex Label Studio Enterprise](https://humansignal.com/blog/bombora-powers-business-with-machine-learning-data-science-heartex-label-studio-enterprise) Zhuoru Lin, Data Scientist at Bombora, the leader in B2B intent data and Heartex customer, sat down with us to discuss how Bombora uses Heartex Label Studio to test and validate new NLP models. Mark Brandsetter December 14, 2021 - [How to Structure, Scale, and Manage a Data Science Team](https://humansignal.com/blog/how-to-structure-scale-and-manage-a-data-science-team) To be fully effective, data scientists need to work with other roles as part of a team. As companies fully embrace data and build their data science departments, it is essential to establish the right processes and workflows first before proceeding to hire people with the right skills needed to implement these processes. Here are some important roles to consider when structuring a data science team. Nikolai Liubimov November 30, 2021 - [Product Updates](https://humansignal.com/blog/categories/product-updates) [Announcing Label Studio Enterprise 2.1.0!](https://humansignal.com/blog/announcing-label-studio-enterprise-2-1-0) The latest updates to Label Studio Enterprise, featuring custom agreement metrics and export snapshots to enhance annotation evaluation for your data science and machine learning projects. Michael Malyuk October 20, 2021