Model Comparison
Compare signature distributions across models to understand their similarities and differences.
Distribution Comparison
Similarity Matrix
| Model | Loading... |
|---|---|
| Loading... | ... |
Historical Data
Track signature changes over time for each model.
Signature History
Loading history data...
Please select a model
Usage
Getting Started
Learn how to use stampr_ai to verify AI model identities in your applications. This guide covers installation, basic usage, and advanced features.
Installation
Install the stampr_ai package using pip:
pip install stampr-aiBasic Usage
Basic Model Verification
Verify that a model is what it claims to be:
from stampr_ai import verify_model
# Verify a model is what it claims to be
result = verify_model("gpt-4o", "OpenAI", api_key)
print(f"Model verified: {result['verified']}")
print(f"Signature: {result.get('matched_hash', 'N/A')}")
print(f"Mode: {result['mode']}")
Verify Against Specific Signature
Check if a model matches a specific signature hash:
# Verify model matches a specific signature hash
result = verify_model("gpt-4o:bede20", "OpenAI", api_key)
if result['verified']:
print("Signature Verified.")
else:
print("Signature mismatch!")
Latest Signature Check
Verify against only the most recent signature:
# Check against the most recent signature only
result = verify_model("gpt-4o:latest", "OpenAI", api_key)
print(f"Latest signature verified: {result['verified']}")
print(f"Signature hash: {result.get('matched_hash')}")
Command Line Usage
You can also use stampr_ai directly from the command line:
Setup Your API Key
# Set your API key as an environment variable
export OPENAI_API_KEY=your_key_hereCommand Examples
# Basic verification
stampr gpt-4o OpenAI
# Verify latest signature only
stampr gpt-4o:latest OpenAI
# Verify specific signature with verbose output
stampr "gpt-4o:bede20" OpenAI --verboseUnderstanding Results
The verification result contains several important fields:
- verified: Boolean indicating if the model signature was verified
- matched_hash: The signature hash that was matched (if verification succeeded)
- mode: The verification mode used (e.g., "latest", "specific", or "any")
- timestamp: When the matched signature was collected
Advanced Examples
Integrating with Your Workflow
import stampr_ai
def safe_model_call(model_name, provider, api_key, prompt):
"""
Verify model before making API calls
"""
# Verify the model first
verification = stampr_ai.verify_model(model_name, provider, api_key)
if not verification['verified']:
raise ValueError(f"Model {model_name} could not be verified!")
print(f"Model verified with signature: {verification['matched_hash']}")
# Proceed with your API call
# ... your model call code here ...
return result
Error Handling
try:
result = stampr_ai.verify_model("gpt-4o", "OpenAI", api_key)
if result['verified']:
print("Model verification successful")
print(f"Signature: {result['matched_hash']}")
else:
print("Model verification failed")
print("This might indicate:")
print("- Model has been updated/changed")
print("- Network connectivity issues")
print("- API key problems")
except Exception as e:
print(f"Error during verification: {e}")
Troubleshooting
Common Issues
Verification fails unexpectedly
This might indicate that the model provider has updated the model. Check the Historical Data tab to see recent changes.
API key errors
Ensure your API key is valid and has sufficient permissions for the model you're trying to verify.
Network timeouts
The verification process requires API calls to both the model provider and our signature database. Check your internet connection.
About stampr_ai
What is stampr_ai?
stampr_ai is a tool for extracting and tracking AI model signatures in a blackbox manner. It monitors model behavior, detects changes, and ensures consistency across different providers.
The tool consists of two main components: a web-based tracker for monitoring model signatures over time, and a Python package for verifying model identities in your applications.
How It Works
- Signature Generation: Our private collector generates unique signatures for AI models by analyzing their output patterns
- Public Verification: The pip package compares your models against our verified signatures to confirm their identity
- Change Detection: The website tracks and displays model changes over time, helping you stay informed about updates
Components
Web Tracker
This tracker page provides:
- Visualization of current model signatures
- Historical tracking of model changes
- Change detection and notifications (soon!)
Python Package
The pip package allows you to:
- Verify model identities programmatically
- Integrate model verification into your workflows
- Check against specific signature versions
pip install stampr-aiContact
Questions? Collaboration ideas? We'd love to hear from you.