Skip to main content

Test Knowledge

Learn how to validate and test your knowledge bases to ensure they provide accurate and relevant information for your AI workflows.

Overview​

Testing your knowledge base ensures:

  • Accurate Retrieval: Content is found when needed
  • Relevant Results: Search returns appropriate information
  • Quality Validation: Content is processed correctly
  • Performance Verification: Search responds quickly
  • Workflow Readiness: Knowledge works effectively in automation

Knowledge Testing Interface​

Accessing the Test Feature​

  1. Navigate to Knowledge: Go to your knowledge base
  2. Test Tab: Click on "Test" or "Manual Run" tab
  3. Test Interface: Access the knowledge retrieval testing interface

Test Interface Components​

Search Parameters Section

  • Query Input: Enter search questions or keywords
  • Max Results: Control number of results returned
  • Search Options: Advanced search parameters
  • Filter Controls: Content filtering options

Results Display

  • Result List: Search results with relevance scores
  • Content Preview: Preview of retrieved content
  • Source Information: Original document and location
  • Metadata Display: Document properties and processing info

Manual Testing Process​

Basic Search Testing​

Step 1: Simple Queries Start with straightforward questions:

Example Queries:
- "What are the office hours?"
- "How do I reset my password?"
- "What is the return policy?"
- "System requirements"

Step 2: Review Results

  • Relevance: Are results related to your query?
  • Completeness: Do results contain the information you need?
  • Accuracy: Is the information correct and up-to-date?
  • Source Quality: Are results from appropriate documents?

Step 3: Complex Queries Test more sophisticated searches:

Complex Query Examples:
- "How do I configure SSL certificates for the web server?"
- "What are the differences between Basic and Premium plans?"
- "Troubleshooting steps for database connection errors"
- "Process for handling customer refund requests"

Query Variations Testing​

Different Phrasings Test the same question in multiple ways:

Password Reset Example:
- "How do I reset my password?"
- "Password reset process"
- "Forgot password procedure"
- "Change password steps"
- "Reset login credentials"

Keyword vs. Natural Language

  • Keywords: "SSL configuration Apache"
  • Natural Language: "How do I set up SSL for Apache web server?"
  • Mixed: "Apache SSL setup process"

Specific vs. General

  • Specific: "Windows 10 installation requirements"
  • General: "system requirements"
  • Context: "What do I need to install this software?"

Advanced Testing Scenarios​

Edge Case Testing​

Misspellings and Typos Test how knowledge handles common errors:

Misspelling Examples:
- "passowrd reset" (password)
- "instalation guide" (installation)
- "configurtion steps" (configuration)

Synonyms and Alternatives Test different terminology:

Synonym Examples:
- "bug" vs "issue" vs "problem"
- "setup" vs "configuration" vs "installation"
- "user" vs "customer" vs "client"

Partial Information Test with incomplete queries:

Partial Query Examples:
- "SSL" (expecting SSL configuration info)
- "refund" (expecting refund policy)
- "backup" (expecting backup procedures)

Content Coverage Testing​

Document Verification Ensure all important documents are indexed:

  1. List Key Documents: Identify critical content
  2. Test Coverage: Search for information from each document
  3. Verify Retrieval: Confirm content is found and accurate
  4. Check Processing: Validate text extraction quality

Topic Coverage Test different subject areas:

  • Technical Topics: API documentation, configuration guides
  • Business Topics: Policies, procedures, guidelines
  • User Topics: How-to guides, troubleshooting, FAQs
  • Reference Topics: Specifications, standards, definitions

Performance Testing​

Response Time

  • Simple Queries: Should return results quickly (under 1 second)
  • Complex Queries: May take longer but should be reasonable
  • Large Result Sets: Test with queries that return many results
  • Concurrent Testing: Test multiple simultaneous queries

Result Quality vs. Speed

  • Accuracy: More results may be more accurate but slower
  • Relevance: Fewer, more relevant results may be faster
  • Optimization: Find the right balance for your use case

Testing Different Content Types​

Structured Documents​

Technical Documentation

Test Queries:
- "API authentication methods"
- "Database schema requirements"
- "System architecture overview"

Policies and Procedures

Test Queries:
- "Employee onboarding process"
- "Expense reimbursement policy"
- "Security incident response"

Unstructured Content​

Text Documents

  • Test extraction from various text formats
  • Verify handling of formatting and structure
  • Check processing of headers and sections

Web Content

  • Test crawled web page content
  • Verify link and navigation content removal
  • Check handling of dynamic content

Mixed Content

  • Test documents with text, tables, and images
  • Verify handling of complex layouts
  • Check extraction from presentations and spreadsheets

Automated Testing​

Test Scripts​

Create repeatable test scenarios:

Test Case Definition

Test Case: Password Reset
Query: "How do I reset my password?"
Expected: Results should include password reset procedures
Success Criteria: Top 3 results contain reset instructions

Regression Testing

  • Baseline Tests: Establish performance baselines
  • Change Impact: Test after content updates
  • Performance Regression: Monitor response time changes
  • Quality Regression: Ensure result quality doesn't degrade

Continuous Testing​

  • Regular Test Runs: Schedule periodic testing
  • Content Update Validation: Test after knowledge updates
  • Performance Monitoring: Track search performance over time
  • Quality Metrics: Monitor result relevance and accuracy

Test Result Analysis​

Quality Metrics​

Relevance Scoring

  • Top Result Relevance: Is the first result what you need?
  • Result Distribution: Are good results spread throughout or clustered?
  • Irrelevant Results: How many results are not useful?

Completeness Assessment

  • Information Coverage: Does the result contain complete information?
  • Missing Information: What important details are missing?
  • Context Adequacy: Is there enough context to understand the result?

Accuracy Validation

  • Factual Correctness: Is the information accurate?
  • Currency: Is the information up-to-date?
  • Source Reliability: Are results from authoritative sources?

Performance Metrics​

Response Time Analysis

  • Average Response Time: Typical query response time
  • Response Time Distribution: Variation in response times
  • Slow Query Identification: Which queries are slowest?
  • Performance Trends: How performance changes over time

Resource Usage

  • Processing Load: Impact on system resources
  • Concurrent User Impact: Performance with multiple users
  • Scaling Behavior: Performance as knowledge base grows

Optimization Based on Testing​

Content Optimization​

Improve Poor Results

  • Add Missing Content: Fill gaps identified during testing
  • Improve Content Quality: Enhance poorly performing documents
  • Update Outdated Information: Refresh stale content
  • Remove Irrelevant Content: Clean up noise in results

Content Structure

  • Better Organization: Improve document structure and headings
  • Keyword Optimization: Ensure important terms are present
  • Context Enhancement: Add context and background information
  • Cross-References: Link related concepts and topics

Search Optimization​

Parameter Tuning

  • Result Limits: Adjust default number of results
  • Relevance Thresholds: Fine-tune relevance scoring
  • Search Algorithms: Optimize search parameters
  • Chunking Strategy: Adjust content chunking for better results

Performance Tuning

  • Index Optimization: Optimize search indexes
  • Caching Strategy: Cache frequent queries
  • Resource Allocation: Adjust system resources
  • Concurrent Limits: Optimize for concurrent users

Integration Testing​

Workflow Integration​

Context Testing Test knowledge in actual workflows:

  1. Create Test Workflow: Simple workflow using knowledge
  2. Run with Test Data: Execute with various inputs
  3. Validate AI Responses: Check AI uses knowledge correctly
  4. Monitor Performance: Ensure workflow performance is acceptable

Real-World Scenarios

  • Customer Support: Test with actual customer questions
  • Document Processing: Test with real documents and tasks
  • Decision Support: Test with actual business scenarios
  • Content Generation: Test knowledge-based content creation

API Testing​

Programmatic Access

  • API Queries: Test knowledge retrieval via API
  • Batch Testing: Test multiple queries efficiently
  • Error Handling: Test API error responses
  • Performance: Test API response times

Troubleshooting Test Issues​

Common Problems​

No Results Found

  • Check Content Processing: Verify documents were indexed
  • Review Query Terms: Try different keywords or phrasings
  • Check Content Quality: Ensure source documents contain the information
  • Validate Permissions: Confirm access to knowledge content

Poor Result Quality

  • Content Issues: Review source document quality
  • Processing Problems: Check text extraction quality
  • Query Refinement: Try more specific or different queries
  • Index Optimization: Consider rebuilding search indexes

Slow Performance

  • Resource Issues: Check system resource availability
  • Index Problems: Optimize or rebuild search indexes
  • Query Complexity: Simplify complex queries
  • Concurrent Load: Reduce concurrent query load

Debugging Steps​

Result Analysis

  1. Review Raw Results: Check what content is actually returned
  2. Check Source Documents: Verify information exists in sources
  3. Analyze Processing Logs: Review content processing logs
  4. Test Alternative Queries: Try different ways to ask the same question

Performance Analysis

  1. Monitor System Resources: Check CPU, memory, and disk usage
  2. Analyze Query Patterns: Identify expensive or slow queries
  3. Review Index Status: Check search index health and optimization
  4. Test System Limits: Identify performance bottlenecks

Best Practices​

Testing Strategy​

  • Regular Testing: Test knowledge bases regularly, not just after creation
  • Comprehensive Coverage: Test all types of content and queries
  • User Perspective: Test from the user's point of view
  • Performance Baselines: Establish and maintain performance baselines

Quality Assurance​

  • Multiple Test Cases: Use diverse test scenarios
  • Edge Case Testing: Test unusual or difficult queries
  • Regression Testing: Ensure changes don't break existing functionality
  • User Feedback: Incorporate actual user experiences

Documentation​

  • Test Results: Document test outcomes and optimizations
  • Known Issues: Track and document known limitations
  • Performance Metrics: Maintain performance history
  • Optimization History: Record changes and their impacts

Thorough testing ensures your knowledge base provides reliable, accurate information for your AI workflows. Continue to Refresh Knowledge to learn about maintaining and updating your knowledge base over time.