diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000..39401da --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,72 @@ +--- +name: Feature Request +about: Suggest an idea for this project +title: 'Implement Automated Release Process' +labels: 'enhancement, ci-cd' +assignees: '' +--- + +## Description +Implement an automated release process using GitHub Actions to streamline the package release workflow. This will ensure consistent and reliable releases while reducing manual intervention. + +## Current Behavior +- Manual version bumping +- Manual changelog updates +- Manual PyPI publishing +- No automated release validation + +## Proposed Solution +Implement a GitHub Actions workflow that will: + +1. **Version Management** + - Automatically detect version changes in pyproject.toml + - Support semantic versioning (major, minor, patch) + - Create version tags automatically + +2. **Changelog Generation** + - Automatically generate CHANGELOG.md entries from commit messages + - Categorize changes (features, fixes, breaking changes) + - Link to relevant PRs and issues + +3. **Release Process** + - Create GitHub releases automatically + - Build and publish to PyPI + - Generate release notes + - Validate package installation + +4. **Quality Checks** + - Run tests before release + - Check code quality + - Verify documentation + - Validate package metadata + +## Implementation Steps +1. Create GitHub Actions workflow file +2. Set up version detection and bumping +3. Implement changelog generation +4. Configure PyPI publishing +5. Add release validation steps +6. Document the release process + +## Required Changes +- Create `.github/workflows/release.yml` +- Update `pyproject.toml` for version management +- Add release documentation +- Configure PyPI secrets in repository + +## Acceptance Criteria +- [ ] Automated version bumping works correctly +- [ ] Changelog is generated automatically +- [ ] Releases are created on GitHub +- [ ] Package is published to PyPI +- [ ] All quality checks pass +- [ ] Documentation is updated +- [ ] Process is documented for contributors + +## Additional Context +This automation will help maintain consistent releases and reduce the chance of human error in the release process. + +## Dependencies +- GitHub Actions +- PyPI account and token +- Poetry for package management \ No newline at end of file diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..1eaf183 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,66 @@ +name: Release + +on: + push: + tags: + - 'v*' + +jobs: + release: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + with: + fetch-depth: 0 + + - name: Set up Python + uses: actions/setup-python@v4 + with: + python-version: '3.9' + + - name: Install Poetry + uses: snok/install-poetry@v1 + with: + version: latest + virtualenvs-create: true + virtualenvs-in-project: true + + - name: Load cached venv + id: cached-poetry-dependencies + uses: actions/cache@v3 + with: + path: .venv + key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }} + + - name: Install dependencies + if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true' + run: poetry install --no-interaction --no-root + + - name: Run tests + run: | + poetry run pytest tests/ + + - name: Build package + run: poetry build + + - name: Create Release + id: create_release + uses: softprops/action-gh-release@v1 + with: + generate_release_notes: true + files: | + dist/* + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + - name: Publish to PyPI + run: | + poetry publish --no-interaction + env: + POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_API_TOKEN }} + + - name: Verify Installation + run: | + VERSION=${GITHUB_REF#refs/tags/v} + pip install --no-cache-dir igel==$VERSION + python -c "import igel; print(igel.__version__)" \ No newline at end of file diff --git a/docs/few_shot_learning.md b/docs/few_shot_learning.md new file mode 100644 index 0000000..33fb710 --- /dev/null +++ b/docs/few_shot_learning.md @@ -0,0 +1,342 @@ +# Few-Shot Learning in igel + +This document describes the few-shot learning capabilities added to igel, addressing GitHub issue #237 "Add Support for Few-Shot Learning". + +## Overview + +Few-shot learning enables machine learning models to learn from very few examples, making it particularly useful when: +- Limited labeled data is available +- New classes need to be learned quickly +- Domain adaptation is required +- Transfer learning is needed + +## Features Implemented + +### 1. Model-Agnostic Meta-Learning (MAML) + +MAML is a meta-learning algorithm that learns to quickly adapt to new tasks with few examples by learning good initial parameters. + +**Key Features:** +- Inner loop adaptation for task-specific learning +- Outer loop meta-update for learning good initial parameters +- Configurable learning rates and training parameters +- Support for both classification and regression tasks + +**Usage:** +```python +from igel.few_shot_learning import MAMLClassifier + +# Initialize MAML +maml = MAMLClassifier( + inner_lr=0.01, # Inner loop learning rate + outer_lr=0.001, # Outer loop learning rate + num_tasks=10, # Tasks per meta-epoch + shots_per_task=5, # Examples per class + inner_steps=5, # Gradient steps for adaptation + meta_epochs=100 # Meta-training epochs +) + +# Train the model +maml.fit(X, y) + +# Make predictions +predictions = maml.predict(X_new) +``` + +### 2. Prototypical Networks + +Prototypical Networks learn a metric space where classification can be performed by computing distances to prototype representations of each class. + +**Key Features:** +- Embedding-based approach for few-shot learning +- Computes class prototypes from support examples +- Uses Euclidean distance for classification +- Efficient for multi-class few-shot problems + +**Usage:** +```python +from igel.few_shot_learning import PrototypicalNetwork + +# Initialize Prototypical Network +proto_net = PrototypicalNetwork( + embedding_dim=64, # Embedding dimension + num_tasks=10, # Tasks per meta-epoch + shots_per_task=5, # Examples per class + meta_epochs=100 # Meta-training epochs +) + +# Train the model +proto_net.fit(X, y) + +# Make predictions +predictions = proto_net.predict(X_new) +``` + +### 3. Domain Adaptation + +Domain adaptation utilities help adapt models trained on a source domain to perform well on a target domain. + +**Supported Methods:** +- **Fine-tuning**: Adapt the entire model to the target domain +- **Domain Adversarial**: Use adversarial training for domain adaptation +- **MAML-based**: Use MAML for domain adaptation + +**Usage:** +```python +from igel.few_shot_learning import DomainAdaptation +from sklearn.ensemble import RandomForestClassifier + +# Create base model +base_model = RandomForestClassifier() + +# Initialize domain adaptation +adapter = DomainAdaptation(base_model) + +# Adapt model to target domain +adapted_model = adapter.adapt_model( + source_X, source_y, # Source domain data + target_X, target_y, # Target domain data + adaptation_method='fine_tune' +) +``` + +### 4. Transfer Learning + +Transfer learning utilities leverage pre-trained models for new tasks. + +**Supported Methods:** +- **Feature Extraction**: Extract features from pre-trained model +- **Fine-tuning**: Fine-tune the entire pre-trained model + +**Usage:** +```python +from igel.few_shot_learning import TransferLearning +import joblib + +# Load pre-trained model +source_model = joblib.load('pretrained_model.joblib') + +# Initialize transfer learning +transfer = TransferLearning(source_model) + +# Create transfer model +transfer_model = transfer.create_transfer_model( + source_model, X_target, y_target, + method='feature_extraction' +) +``` + +## CLI Commands + +### Few-Shot Learning Training + +```bash +igel few-shot-learn \ + --data_path=data/train.csv \ + --yaml_path=config.yaml \ + --n_way=3 \ + --k_shot=5 \ + --n_query=5 +``` + +### Domain Adaptation + +```bash +igel domain-adapt \ + --source_data=source.csv \ + --target_data=target.csv \ + --method=fine_tune \ + --output_model=adapted_model.joblib +``` + +### Transfer Learning + +```bash +igel transfer-learn \ + --source_model=pretrained.joblib \ + --target_data=new_data.csv \ + --method=feature_extraction \ + --output_model=transfer_model.joblib +``` + +## Configuration Files + +### Few-Shot Learning Configuration + +```yaml +# few_shot_config.yaml +dataset: + missing_values: "drop" + categorical_encoding: "label" + scaling: "standard" + random_numbers: + generate_reproducible: true + seed: 42 + +model: + type: "few_shot_learning" + algorithm: "MAML" # or "PrototypicalNetwork" + + arguments: + # MAML parameters + inner_lr: 0.01 + outer_lr: 0.001 + num_tasks: 10 + shots_per_task: 5 + inner_steps: 5 + meta_epochs: 100 + + # Prototypical Network parameters + # embedding_dim: 64 + +target: ["target"] +``` + +## Utility Functions + +### Creating Few-Shot Tasks + +```python +from igel.few_shot_learning import create_few_shot_dataset + +# Create few-shot tasks +tasks = create_few_shot_dataset( + X, y, + n_way=3, # Number of classes per task + k_shot=5, # Examples per class for support + n_query=5 # Examples per class for query +) +``` + +### Evaluating Few-Shot Models + +```python +from igel.few_shot_learning import evaluate_few_shot_model + +# Evaluate model on few-shot tasks +results = evaluate_few_shot_model(model, tasks) + +print(f"Mean accuracy: {results['mean_accuracy']:.4f}") +print(f"Std accuracy: {results['std_accuracy']:.4f}") +``` + +## Examples + +### Complete Example: MAML Training + +```python +import pandas as pd +from igel.few_shot_learning import MAMLClassifier, create_few_shot_dataset, evaluate_few_shot_model + +# Load data +data = pd.read_csv('data.csv') +X = data.drop(columns=['target']).values +y = data['target'].values + +# Initialize MAML +maml = MAMLClassifier( + inner_lr=0.01, + outer_lr=0.001, + num_tasks=10, + shots_per_task=5, + inner_steps=5, + meta_epochs=100 +) + +# Train model +maml.fit(X, y) + +# Create evaluation tasks +tasks = create_few_shot_dataset(X, y, n_way=2, k_shot=5, n_query=5) + +# Evaluate +results = evaluate_few_shot_model(maml, tasks) +print(f"Mean accuracy: {results['mean_accuracy']:.4f}") + +# Save model +import joblib +joblib.dump(maml, 'maml_model.joblib') +``` + +### Domain Adaptation Example + +```python +import pandas as pd +from igel.few_shot_learning import DomainAdaptation +from sklearn.ensemble import RandomForestClassifier + +# Load source and target data +source_data = pd.read_csv('source.csv') +target_data = pd.read_csv('target.csv') + +X_source = source_data.drop(columns=['target']).values +y_source = source_data['target'].values +X_target = target_data.drop(columns=['target']).values +y_target = target_data['target'].values + +# Create and train base model +base_model = RandomForestClassifier() +base_model.fit(X_source, y_source) + +# Perform domain adaptation +adapter = DomainAdaptation(base_model) +adapted_model = adapter.adapt_model( + X_source, y_source, X_target, y_target, + adaptation_method='fine_tune' +) + +# Save adapted model +import joblib +joblib.dump(adapted_model, 'adapted_model.joblib') +``` + +## Best Practices + +### 1. Data Preparation +- Ensure sufficient class diversity in your dataset +- Balance the number of examples per class +- Preprocess data consistently across domains + +### 2. Hyperparameter Tuning +- Start with default parameters and adjust based on results +- Use cross-validation for hyperparameter selection +- Monitor meta-loss during training + +### 3. Evaluation +- Use multiple few-shot tasks for evaluation +- Report mean and standard deviation of accuracy +- Compare against baseline methods + +### 4. Model Selection +- Use MAML for complex adaptation scenarios +- Use Prototypical Networks for metric-based learning +- Use domain adaptation when source and target domains differ + +## Limitations and Considerations + +1. **Computational Cost**: Meta-learning can be computationally expensive +2. **Data Requirements**: Still requires sufficient data for meta-training +3. **Hyperparameter Sensitivity**: Performance can be sensitive to hyperparameters +4. **Task Similarity**: Assumes tasks are related during meta-training + +## Future Enhancements + +Potential improvements for future versions: +- Support for regression tasks in MAML +- More sophisticated domain adaptation methods +- Integration with deep learning frameworks +- Automated hyperparameter optimization +- Support for multi-modal few-shot learning + +## References + +1. Finn, C., Abbeel, P., & Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. +2. Snell, J., Swersky, K., & Zemel, R. (2017). Prototypical networks for few-shot learning. +3. Ganin, Y., & Lempitsky, V. (2015). Unsupervised domain adaptation by backpropagation. + +## Support + +For issues and questions related to few-shot learning in igel: +- Check the examples in `examples/few_shot_learning_demo.py` +- Review the configuration examples +- Open an issue on the GitHub repository \ No newline at end of file diff --git a/examples/auto-ml/igel.yaml b/examples/auto-ml/igel.yaml index 6a9e804..9d692b3 100644 --- a/examples/auto-ml/igel.yaml +++ b/examples/auto-ml/igel.yaml @@ -1,5 +1,3 @@ - - model: type: ImageClassification arguments: @@ -7,3 +5,8 @@ model: training: epochs: 10 + +dataset: + source_type: sql + connection_string: "sqlite:///mydb.sqlite" + sql_query: "SELECT * FROM my_table" diff --git a/examples/feature_engineering_demo.py b/examples/feature_engineering_demo.py new file mode 100644 index 0000000..84ea3d5 Binary files /dev/null and b/examples/feature_engineering_demo.py differ diff --git a/examples/feature_engineering_example.yaml b/examples/feature_engineering_example.yaml new file mode 100644 index 0000000..ba03d3d Binary files /dev/null and b/examples/feature_engineering_example.yaml differ diff --git a/examples/few_shot_learning_demo.py b/examples/few_shot_learning_demo.py new file mode 100644 index 0000000..723fa2a --- /dev/null +++ b/examples/few_shot_learning_demo.py @@ -0,0 +1,346 @@ +#!/usr/bin/env python3 +""" +Few-shot Learning Demo for igel + +This script demonstrates the few-shot learning capabilities of igel, including: +1. Model-Agnostic Meta-Learning (MAML) +2. Prototypical Networks +3. Domain Adaptation +4. Transfer Learning + +Usage: + python examples/few_shot_learning_demo.py +""" + +import numpy as np +import pandas as pd +import joblib +from sklearn.datasets import make_classification, make_regression +from sklearn.model_selection import train_test_split +from sklearn.preprocessing import StandardScaler +from sklearn.metrics import accuracy_score, classification_report +import os + +# Import igel few-shot learning modules +from igel.few_shot_learning import ( + MAMLClassifier, + PrototypicalNetwork, + DomainAdaptation, + TransferLearning, + create_few_shot_dataset, + evaluate_few_shot_model +) + +def create_sample_data(n_samples=1000, n_features=20, n_classes=5, random_state=42): + """Create sample classification data for demonstration.""" + X, y = make_classification( + n_samples=n_samples, + n_features=n_features, + n_classes=n_classes, + n_clusters_per_class=1, + n_redundant=5, + n_informative=10, + random_state=random_state + ) + + # Create DataFrame + feature_names = [f'feature_{i}' for i in range(n_features)] + df = pd.DataFrame(X, columns=feature_names) + df['target'] = y + + return df + +def create_domain_data(base_data, domain_shift=0.5, random_state=42): + """Create domain-shifted data for domain adaptation demo.""" + np.random.seed(random_state) + + # Add domain shift by modifying features + X = base_data.drop(columns=['target']).values + y = base_data['target'].values + + # Add noise and shift to create domain shift + X_shifted = X + np.random.normal(0, domain_shift, X.shape) + + # Create new DataFrame + feature_names = [f'feature_{i}' for i in range(X.shape[1])] + df_shifted = pd.DataFrame(X_shifted, columns=feature_names) + df_shifted['target'] = y + + return df_shifted + +def demo_maml(): + """Demonstrate MAML (Model-Agnostic Meta-Learning).""" + print("\n" + "="*50) + print("DEMO: Model-Agnostic Meta-Learning (MAML)") + print("="*50) + + # Create sample data + data = create_sample_data(n_samples=500, n_classes=4) + X = data.drop(columns=['target']).values + y = data['target'].values + + print(f"Dataset shape: {X.shape}") + print(f"Number of classes: {len(np.unique(y))}") + + # Initialize MAML classifier + maml = MAMLClassifier( + inner_lr=0.01, + outer_lr=0.001, + num_tasks=5, + shots_per_task=3, + inner_steps=3, + meta_epochs=20 # Reduced for demo + ) + + print("\nTraining MAML model...") + maml.fit(X, y) + + # Create few-shot tasks for evaluation + tasks = create_few_shot_dataset(X, y, n_way=2, k_shot=3, n_query=3) + + # Evaluate on few-shot tasks + results = evaluate_few_shot_model(maml, tasks) + + print(f"\nMAML Results:") + print(f"Mean accuracy: {results['mean_accuracy']:.4f}") + print(f"Std accuracy: {results['std_accuracy']:.4f}") + + # Save model + joblib.dump(maml, "maml_model.joblib") + print("MAML model saved as 'maml_model.joblib'") + + return maml + +def demo_prototypical_networks(): + """Demonstrate Prototypical Networks.""" + print("\n" + "="*50) + print("DEMO: Prototypical Networks") + print("="*50) + + # Create sample data + data = create_sample_data(n_samples=500, n_classes=4) + X = data.drop(columns=['target']).values + y = data['target'].values + + print(f"Dataset shape: {X.shape}") + print(f"Number of classes: {len(np.unique(y))}") + + # Initialize Prototypical Network + proto_net = PrototypicalNetwork( + embedding_dim=32, + num_tasks=5, + shots_per_task=3, + meta_epochs=20 # Reduced for demo + ) + + print("\nTraining Prototypical Network...") + proto_net.fit(X, y) + + # Create few-shot tasks for evaluation + tasks = create_few_shot_dataset(X, y, n_way=2, k_shot=3, n_query=3) + + # Evaluate on few-shot tasks + results = evaluate_few_shot_model(proto_net, tasks) + + print(f"\nPrototypical Network Results:") + print(f"Mean accuracy: {results['mean_accuracy']:.4f}") + print(f"Std accuracy: {results['std_accuracy']:.4f}") + + # Save model + joblib.dump(proto_net, "prototypical_network.joblib") + print("Prototypical Network model saved as 'prototypical_network.joblib'") + + return proto_net + +def demo_domain_adaptation(): + """Demonstrate Domain Adaptation.""" + print("\n" + "="*50) + print("DEMO: Domain Adaptation") + print("="*50) + + # Create source and target domain data + source_data = create_sample_data(n_samples=300, n_classes=3) + target_data = create_domain_data(source_data, domain_shift=0.8) + + X_source = source_data.drop(columns=['target']).values + y_source = source_data['target'].values + X_target = target_data.drop(columns=['target']).values + y_target = target_data['target'].values + + print(f"Source domain shape: {X_source.shape}") + print(f"Target domain shape: {X_target.shape}") + + # Create a base model + from sklearn.ensemble import RandomForestClassifier + base_model = RandomForestClassifier(n_estimators=50, random_state=42) + + # Train on source domain + print("\nTraining base model on source domain...") + base_model.fit(X_source, y_source) + + # Evaluate on source domain + source_pred = base_model.predict(X_source) + source_acc = accuracy_score(y_source, source_pred) + print(f"Source domain accuracy: {source_acc:.4f}") + + # Evaluate on target domain (before adaptation) + target_pred_before = base_model.predict(X_target) + target_acc_before = accuracy_score(y_target, target_pred_before) + print(f"Target domain accuracy (before adaptation): {target_acc_before:.4f}") + + # Perform domain adaptation + print("\nPerforming domain adaptation...") + adapter = DomainAdaptation(base_model) + adapted_model = adapter.adapt_model( + X_source, y_source, X_target, y_target, + adaptation_method='fine_tune' + ) + + # Evaluate on target domain (after adaptation) + target_pred_after = adapted_model.predict(X_target) + target_acc_after = accuracy_score(y_target, target_pred_after) + print(f"Target domain accuracy (after adaptation): {target_acc_after:.4f}") + + print(f"Improvement: {target_acc_after - target_acc_before:.4f}") + + # Save adapted model + joblib.dump(adapted_model, "adapted_model.joblib") + print("Adapted model saved as 'adapted_model.joblib'") + + return adapted_model + +def demo_transfer_learning(): + """Demonstrate Transfer Learning.""" + print("\n" + "="*50) + print("DEMO: Transfer Learning") + print("="*50) + + # Create source and target data + source_data = create_sample_data(n_samples=500, n_classes=4) + target_data = create_sample_data(n_samples=100, n_classes=3, random_state=123) + + X_source = source_data.drop(columns=['target']).values + y_source = source_data['target'].values + X_target = target_data.drop(columns=['target']).values + y_target = target_data['target'].values + + print(f"Source data shape: {X_source.shape}") + print(f"Target data shape: {X_target.shape}") + + # Train source model + from sklearn.ensemble import RandomForestClassifier + source_model = RandomForestClassifier(n_estimators=100, random_state=42) + source_model.fit(X_source, y_source) + + print(f"Source model accuracy: {source_model.score(X_source, y_source):.4f}") + + # Perform transfer learning + print("\nPerforming transfer learning...") + transfer = TransferLearning(source_model) + + # Method 1: Feature extraction + print("\n1. Feature extraction method:") + transfer_model_fe = transfer.create_transfer_model( + source_model, X_target, y_target, method='feature_extraction' + ) + fe_acc = transfer_model_fe.score(X_target, y_target) + print(f"Feature extraction accuracy: {fe_acc:.4f}") + + # Method 2: Fine-tuning + print("\n2. Fine-tuning method:") + transfer_model_ft = transfer.create_transfer_model( + source_model, X_target, y_target, method='fine_tuning' + ) + ft_acc = transfer_model_ft.score(X_target, y_target) + print(f"Fine-tuning accuracy: {ft_acc:.4f}") + + # Save transfer models + joblib.dump(transfer_model_fe, "transfer_model_fe.joblib") + joblib.dump(transfer_model_ft, "transfer_model_ft.joblib") + print("\nTransfer learning models saved") + + return transfer_model_fe, transfer_model_ft + +def demo_few_shot_tasks(): + """Demonstrate few-shot task creation and evaluation.""" + print("\n" + "="*50) + print("DEMO: Few-shot Task Creation") + print("="*50) + + # Create sample data + data = create_sample_data(n_samples=400, n_classes=6) + X = data.drop(columns=['target']).values + y = data['target'].values + + print(f"Dataset shape: {X.shape}") + print(f"Number of classes: {len(np.unique(y))}") + + # Create few-shot tasks + tasks = create_few_shot_dataset( + X, y, + n_way=3, # 3 classes per task + k_shot=5, # 5 examples per class for support + n_query=5 # 5 examples per class for query + ) + + print(f"Created {len(tasks)} few-shot tasks") + + # Analyze task distribution + for i, (support_X, support_y, query_X, query_y) in enumerate(tasks[:3]): + print(f"\nTask {i+1}:") + print(f" Support set: {support_X.shape}, classes: {np.unique(support_y)}") + print(f" Query set: {query_X.shape}, classes: {np.unique(query_y)}") + + return tasks + +def main(): + """Run all few-shot learning demonstrations.""" + print("IGEL Few-shot Learning Demo") + print("="*60) + print("This demo showcases the few-shot learning capabilities of igel") + print("including MAML, Prototypical Networks, Domain Adaptation, and Transfer Learning.") + + # Create output directory + os.makedirs("few_shot_demo_output", exist_ok=True) + + try: + # Demo 1: MAML + maml_model = demo_maml() + + # Demo 2: Prototypical Networks + proto_model = demo_prototypical_networks() + + # Demo 3: Domain Adaptation + adapted_model = demo_domain_adaptation() + + # Demo 4: Transfer Learning + transfer_models = demo_transfer_learning() + + # Demo 5: Few-shot Task Creation + tasks = demo_few_shot_tasks() + + print("\n" + "="*60) + print("DEMO COMPLETED SUCCESSFULLY!") + print("="*60) + print("\nGenerated files:") + print("- maml_model.joblib") + print("- prototypical_network.joblib") + print("- adapted_model.joblib") + print("- transfer_model_fe.joblib") + print("- transfer_model_ft.joblib") + + print("\nNext steps:") + print("1. Use these models for predictions on new data") + print("2. Experiment with different hyperparameters") + print("3. Try the CLI commands:") + print(" - igel few-shot-learn --data_path=your_data.csv --yaml_path=config.yaml") + print(" - igel domain-adapt --source_data=source.csv --target_data=target.csv") + print(" - igel transfer-learn --source_model=model.joblib --target_data=new_data.csv") + + except Exception as e: + print(f"\nError during demo: {e}") + import traceback + traceback.print_exc() + +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/examples/few_shot_learning_example.yaml b/examples/few_shot_learning_example.yaml new file mode 100644 index 0000000..2ced5dc --- /dev/null +++ b/examples/few_shot_learning_example.yaml @@ -0,0 +1,40 @@ +# Few-shot Learning Configuration Example +# This example shows how to configure igel for few-shot learning tasks + +# Dataset configuration +dataset: + # Data preprocessing options + missing_values: "drop" # or "fill" + categorical_encoding: "label" # or "onehot" + scaling: "standard" # or "minmax", "robust" + + # Few-shot learning specific options + random_numbers: + generate_reproducible: true + seed: 42 + +# Model configuration for few-shot learning +model: + type: "few_shot_learning" # New model type for few-shot learning + algorithm: "MAML" # Options: "MAML", "PrototypicalNetwork" + + # MAML-specific arguments + arguments: + # Learning rates + inner_lr: 0.01 # Learning rate for inner loop adaptation + outer_lr: 0.001 # Learning rate for outer loop meta-update + + # Task configuration + num_tasks: 10 # Number of tasks to sample per meta-epoch + shots_per_task: 5 # Number of examples per class (k-shot learning) + inner_steps: 5 # Number of gradient steps for inner loop + meta_epochs: 100 # Number of meta-training epochs + + # For Prototypical Networks, you can also use: + # embedding_dim: 64 # Dimension of the embedding space + +# Target column(s) to predict +target: ["target"] + +# Example usage: +# igel few-shot-learn --data_path=data/train.csv --yaml_path=examples/few_shot_learning_example.yaml \ No newline at end of file diff --git a/examples/sample_config.json b/examples/sample_config.json new file mode 100644 index 0000000..3cbb030 --- /dev/null +++ b/examples/sample_config.json @@ -0,0 +1,13 @@ + +{ + "input": "data/dataset.csv", + "target": "label", + "model": { + "type": "classification", + "algorithm": "RandomForestClassifier", + "params": { + "n_estimators": 100, + "max_depth": 5 + } + } +} diff --git a/examples/sample_config.yaml b/examples/sample_config.yaml new file mode 100644 index 0000000..31b590c --- /dev/null +++ b/examples/sample_config.yaml @@ -0,0 +1,14 @@ +# Unique contribution: HeerakKashyap 2024-05-18 15:30 +# Path to the dataset +input: data/dataset.csv + +# Target column for prediction +target: label + +# Model configuration +model: + type: classification + algorithm: RandomForestClassifier + params: + n_estimators: 100 + max_depth: 5 \ No newline at end of file diff --git a/igel/.dockerignore b/igel/.dockerignore new file mode 100644 index 0000000..e682d86 --- /dev/null +++ b/igel/.dockerignore @@ -0,0 +1,2 @@ +.git +model_results \ No newline at end of file diff --git a/igel/.editorconfig b/igel/.editorconfig new file mode 100644 index 0000000..d635cdf --- /dev/null +++ b/igel/.editorconfig @@ -0,0 +1,28 @@ +# EditorConfig helps maintain consistent coding styles for multiple developers working on the same project +# See http://editorconfig.org + +root = true + +[*] +charset = utf-8 +end_of_line = lf +insert_final_newline = true +indent_style = space +indent_size = 4 +trim_trailing_whitespace = true + +[*.md] +trim_trailing_whitespace = false + +[*.rst] +trim_trailing_whitespace = false + +[*.bat] +indent_style = tab +end_of_line = crlf + +[LICENSE] +insert_final_newline = false + +[Makefile] +indent_style = tab diff --git a/igel/.github/FUNDING.yml b/igel/.github/FUNDING.yml new file mode 100644 index 0000000..b3b7f70 --- /dev/null +++ b/igel/.github/FUNDING.yml @@ -0,0 +1,3 @@ +# These are supported funding model platforms + +github: nidhaloff diff --git a/igel/.github/ISSUE_TEMPLATE.md b/igel/.github/ISSUE_TEMPLATE.md new file mode 100644 index 0000000..e53fd73 --- /dev/null +++ b/igel/.github/ISSUE_TEMPLATE.md @@ -0,0 +1,15 @@ +* igel version: +* Python version: +* Operating System: + +### Description + +Describe what you were trying to get done. +Tell us what happened, what went wrong, and what you expected to happen. + +### What I Did + +``` +Paste the command(s) you ran and the output. +If there was a crash, please include the traceback here. +``` diff --git a/igel/.github/ISSUE_TEMPLATE/bug_report.md b/igel/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000..2a352ca --- /dev/null +++ b/igel/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,31 @@ +--- +name: Bug report +about: Create a report to help us improve +title: "[BUG] " +labels: bug +assignees: '' + +--- + +**Describe the bug** +A clear and concise description of what the bug is. + +**To Reproduce** +Steps to reproduce the behavior: +1. Go to '...' +2. Run '...' +3. See error + +**Expected behavior** +A clear and concise description of what you expected to happen. + +**Screenshots** +If applicable, add screenshots to help explain your problem. + +**Environment (please complete the following information):** + - OS: [e.g. Windows, MacOS, Linux] + - Python version: [e.g. 3.8] + - igel version: [e.g. 0.7.0] + +**Additional context** +Add any other context about the problem here. diff --git a/igel/.github/ISSUE_TEMPLATE/feature_request.md b/igel/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 0000000..f83b539 --- /dev/null +++ b/igel/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,20 @@ +--- +name: Feature request +about: Suggest an idea for this project +title: "[FEATURE] " +labels: enhancement +assignees: '' + +--- + +**Is your feature request related to a problem? Please describe.** +A clear and concise description of what the problem is. + +**Describe the solution you'd like** +A clear and concise description of what you want to happen. + +**Describe alternatives you've considered** +A clear and concise description of any alternative solutions or features you've considered. + +**Additional context** +Add any other context or screenshots about the feature request here. diff --git a/igel/.github/pull_request_template.md b/igel/.github/pull_request_template.md new file mode 100644 index 0000000..a8d51b4 --- /dev/null +++ b/igel/.github/pull_request_template.md @@ -0,0 +1,26 @@ +# Pull Request + +## Description + +Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. + +Fixes #(issue) + +## Type of change + +Please delete options that are not relevant. + +- [ ] Bug fix +- [ ] New feature +- [ ] Breaking change +- [ ] Documentation update + +## Checklist + +- [ ] My code follows the style guidelines of this project +- [ ] I have performed a self-review of my own code +- [ ] I have commented my code, particularly in hard-to-understand areas +- [ ] I have made corresponding changes to the documentation +- [ ] My changes generate no new warnings +- [ ] I have added tests that prove my fix is effective or that my feature works +- [ ] New and existing unit tests pass locally with my changes diff --git a/igel/.github/workflows/build.yml b/igel/.github/workflows/build.yml new file mode 100644 index 0000000..49554c4 --- /dev/null +++ b/igel/.github/workflows/build.yml @@ -0,0 +1,58 @@ +name: build + +on: [push, pull_request] + +jobs: + build: + runs-on: ubuntu-latest + strategy: + matrix: + python-version: ["3.7", "3.8", "3.9", "3.10"] + + steps: + - uses: actions/checkout@v2 + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v2.2.1 + with: + python-version: ${{ matrix.python-version }} + + - name: Install poetry + run: make download-poetry + + - name: Set up cache + uses: actions/cache@v2.1.3 + with: + path: .venv + key: venv-${{ matrix.python-version }}-${{ hashFiles('pyproject.toml') }}-${{ hashFiles('poetry.lock') }} + - name: Install dependencies + run: | + source "$HOME/.poetry/env" + poetry config virtualenvs.in-project true + poetry install + + - name: Run safety checks + run: | + source "$HOME/.poetry/env" + STRICT=1 make check-safety + + - name: Run style checks + run: | + source "$HOME/.poetry/env" + STRICT=1 make check-style + poetry run black --check . + + - name: Run tests + run: | + source "$HOME/.poetry/env" + make test + + - name: Run coverage + run: | + source "$HOME/.poetry/env" + poetry run coverage run -m pytest + poetry run coverage xml + + - name: Upload coverage to Codecov + uses: codecov/codecov-action@v3 + with: + files: ./coverage.xml diff --git a/igel/.github/workflows/release.yml b/igel/.github/workflows/release.yml new file mode 100644 index 0000000..59ef37d --- /dev/null +++ b/igel/.github/workflows/release.yml @@ -0,0 +1,39 @@ +name: Publish Python 🐍 distribution 📦 to PyPI + +on: + push: + tags: + - 'v*.*.*' + +jobs: + build-and-publish: + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v2 + + - name: Set up Python + uses: actions/setup-python@v2 + with: + python-version: '3.8' + + - name: Install Poetry + run: | + curl -sSL https://install.python-poetry.org | python3 - + + - name: Install dependencies + run: | + source $HOME/.poetry/env + poetry install + + - name: Build package + run: | + source $HOME/.poetry/env + poetry build + + - name: Publish to PyPI + env: + POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_API_TOKEN }} + run: | + source $HOME/.poetry/env + poetry publish --no-interaction --username __token__ --password $POETRY_PYPI_TOKEN_PYPI diff --git a/igel/.gitignore b/igel/.gitignore new file mode 100644 index 0000000..1df7859 --- /dev/null +++ b/igel/.gitignore @@ -0,0 +1,118 @@ +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + + +# examples: +examples/model_results +examples/pre-release-example +examples/serving_models + +!examples/*.yaml +!examples/*.yml +!examples/*.json + +# Distribution / packaging +.Python +env/ +.idea/ +idea/ +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +*.egg-info/ +.installed.cfg +*.egg + +# PyInstaller +# Usually these files are written by a python script from a template +# before PyInstaller builds the exe, so as to inject date/other infos into it. +*.manifest +*.spec + +# Installer logs +pip-log.txt +pip-delete-this-directory.txt + +# Unit test / coverage reports +htmlcov/ +.tox/ +.coverage +.coverage.* +.cache +nosetests.xml +coverage.xml +*.cover +.hypothesis/ +.pytest_cache/ + +# Translations +*.mo +*.pot + +# Django stuff: +*.log +local_settings.py + +# Flask stuff: +instance/ +.webassets-cache + +# Scrapy stuff: +.scrapy + +# Sphinx documentation +docs/_build/ + +# PyBuilder +target/ + +# Jupyter Notebook +.ipynb_checkpoints + +# pyenv +.python-version + +# celery beat schedule file +celerybeat-schedule + +# SageMath parsed files +*.sage.py + +# dotenv +.env + +# virtualenv +.venv +venv/ +ENV/ + +# Spyder project settings +.spyderproject +.spyproject + +# Rope project settings +.ropeproject + +# mkdocs documentation +/site + +# mypy +.mypy_cache/ + +# IDE settings +.vscode/ + diff --git a/igel/.pre-commit-config.yaml b/igel/.pre-commit-config.yaml new file mode 100644 index 0000000..aa3eb25 --- /dev/null +++ b/igel/.pre-commit-config.yaml @@ -0,0 +1,35 @@ +default_language_version: + python: python3.8 + +default_stages: [commit, push] + +repos: + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v2.5.0 + hooks: + - id: check-yaml + - id: end-of-file-fixer + + - repo: local + hooks: + - id: pyupgrade + name: pyupgrade + entry: poetry run pyupgrade --py37-plus + types: [python] + language: system + + - repo: local + hooks: + - id: isort + name: isort + entry: poetry run isort --settings-path pyproject.toml + types: [python] + language: system + + - repo: local + hooks: + - id: black + name: black + entry: poetry run black --config pyproject.toml + types: [python] + language: system diff --git a/igel/.readthedocs.yaml b/igel/.readthedocs.yaml new file mode 100644 index 0000000..90dd877 --- /dev/null +++ b/igel/.readthedocs.yaml @@ -0,0 +1,14 @@ + +# File: .readthedocs.yaml + +version: 2 + +# Build from the docs/ directory with Sphinx +sphinx: + configuration: docs/conf.py + +# Explicitly set the version of Python and its requirements +python: + version: 3.7 + install: + - requirements: docs/requirements_docs.txt diff --git a/igel/AUTHORS.rst b/igel/AUTHORS.rst new file mode 100644 index 0000000..b2c0d0a --- /dev/null +++ b/igel/AUTHORS.rst @@ -0,0 +1,13 @@ +======= +Credits +======= + +Development Lead +---------------- + +* Nidhal Baccouri + +Contributors +------------ + +Check the list on github ;) diff --git a/igel/CHANGELOG.md b/igel/CHANGELOG.md new file mode 100644 index 0000000..7db1332 --- /dev/null +++ b/igel/CHANGELOG.md @@ -0,0 +1,18 @@ +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [Unreleased] + +## [1.0.0] - YYYY-MM-DD +### Added +- Initial release of igel. + +### Changed +- (List any major changes here) + +### Fixed +- (List any bug fixes here) diff --git a/igel/CODE_OF_CONDUCT.md b/igel/CODE_OF_CONDUCT.md new file mode 100644 index 0000000..8454eaa --- /dev/null +++ b/igel/CODE_OF_CONDUCT.md @@ -0,0 +1,46 @@ +# Contributor Covenant Code of Conduct + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation. + +We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment for our community include: + +* Demonstrating empathy and kindness toward other people +* Being respectful of differing opinions, viewpoints, and experiences +* Giving and gracefully accepting constructive feedback +* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience +* Focusing on what is best not just for us as individuals, but for the overall community + +Examples of unacceptable behavior include: + +* The use of sexualized language or imagery, and sexual attention or advances of any kind +* Trolling, insulting or derogatory comments, and personal or political attacks +* Public or private harassment +* Publishing others' private information, such as a physical or email address, without their explicit permission +* Other conduct which could reasonably be considered inappropriate in a professional setting + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at [kashyapheerk@gmail.com]. All complaints will be reviewed and investigated promptly and fairly. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.1, available at https://www.contributor-covenant.org/version/2/1/code_of_conduct.html + +[homepage]: https://www.contributor-covenant.org + +For answers to common questions about this code of conduct, see +https://www.contributor-covenant.org/faq diff --git a/igel/CONTRIBUTING.rst b/igel/CONTRIBUTING.rst new file mode 100644 index 0000000..352a986 --- /dev/null +++ b/igel/CONTRIBUTING.rst @@ -0,0 +1,141 @@ +.. highlight:: shell + +============ +Contributing +============ + +Contributions are welcome, and they are greatly appreciated! Every little bit +helps, and credit will always be given. + +You can contribute in many ways: + +Types of Contributions +---------------------- + +Report Bugs +~~~~~~~~~~~ + +Report bugs at https://github.com/nidhaloff/igel/issues. + +If you are reporting a bug, please include: + +* Your operating system name and version. +* Any details about your local setup that might be helpful in troubleshooting. +* Detailed steps to reproduce the bug. + +Fix Bugs +~~~~~~~~ + +Look through the GitHub issues for bugs. Anything tagged with "bug" and "help +wanted" is open to whoever wants to implement it. + +Implement Features +~~~~~~~~~~~~~~~~~~ + +Look through the GitHub issues for features. Anything tagged with "enhancement" +and "help wanted" is open to whoever wants to implement it. + +Write Documentation +~~~~~~~~~~~~~~~~~~~ + +igel could always use more documentation, whether as part of the +official igel docs, in docstrings, or even on the web in blog posts, +articles, and such. + +Submit Feedback +~~~~~~~~~~~~~~~ + +The best way to send feedback is to file an issue at https://github.com/nidhaloff/igel/issues. + +If you are proposing a feature: + +* Explain in detail how it would work. +* Keep the scope as narrow as possible, to make it easier to implement. +* Remember that this is a volunteer-driven project, and that contributions + are welcome :) + +Get Started! +------------ + +Ready to contribute? Here's how to set up `igel` for local development. + +1. Fork the `igel` repo on GitHub. +2. Clone your fork locally:: + + $ git clone git@github.com:nidhaloff/igel.git + +3. Install your local copy into a virtualenv. Assuming you have poetry (https://pypi.org/project/poetry/) installed, this is how you set up your fork for local development:: + + $ cd igel/ + $ poetry shell + $ poetry update + $ poetry install + +4. Create a branch for local development:: + + $ git checkout -b name-of-your-bugfix-or-feature + + Now you can make your changes locally. + +5. When you're done making changes, check that your changes pass flake8 and the + tests, including testing other Python versions with tox:: + + $ make test + + To get poetry, just run pip install poetry. + +6. Commit your changes and push your branch to GitHub:: + + $ git add . + $ git commit -m "Your detailed description of your changes." + $ git push origin name-of-your-bugfix-or-feature + +7. Submit a pull request through the GitHub website. + +Pull Request Guidelines +----------------------- + +Before you submit a pull request, check that it meets these guidelines: + +1. The pull request should include tests. +2. If the pull request adds functionality, the docs should be updated. Put + your new functionality into a function with a docstring, and add the + feature to the list in README.rst. + +## Releasing to PyPI + +To release a new version: +1. Bump the version in `pyproject.toml`. +2. Commit and push your changes. +3. Create a new tag (e.g., `git tag v1.2.3 && git push origin v1.2.3`). +4. The GitHub Actions workflow will automatically build and publish the package to PyPI. + +## 👋 For First-Time Contributors + +Welcome! We're excited to have you contribute to igel. Here are some tips to help you get started: + +- **Read the README and existing documentation** to understand the project's purpose and structure. +- **Look for issues labeled [`good first issue`](https://github.com/nidhaloff/igel/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)** — these are great entry points for new contributors. +- **Ask questions!** If you're unsure about anything, open an issue or comment on an existing one. +- **Fork the repository** and create a new branch for your changes. +- **Follow the code style guidelines** (see below). +- **Write clear commit messages** and link your pull request to the relevant issue (e.g., `Closes #143`). +- **Be respectful and collaborative** — we value every contribution! + +### Useful Resources + +- [GitHub's guide to contributing to open source](https://opensource.guide/how-to-contribute/) +- [How to create a Pull Request](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests) +- [Semantic commit messages](https://www.conventionalcommits.org/en/v1.0.0/) + +Thank you for helping make igel better! + +## Exporting a Trained Model to ONNX + +You can export a trained scikit-learn model to ONNX format using the CLI: + +```sh +python -m igel export --model_path path/to/your/model.joblib +``` + +This will create an ONNX file (e.g., `model.onnx`) in your results directory for interoperability with other tools. diff --git a/igel/Dockerfile b/igel/Dockerfile new file mode 100644 index 0000000..6c88d3c --- /dev/null +++ b/igel/Dockerfile @@ -0,0 +1,22 @@ +FROM python:3.8 + +RUN mkdir /data && \ + mkdir /igel + +COPY requirements.txt /igel/requirements.txt +RUN pip install -r /igel/requirements.txt + +COPY assets /igel/assets +COPY docs /igel/docs +COPY igel /igel/igel +COPY setup.cfg /igel/setup.cfg +COPY setup.py /igel/setup.py +COPY HISTORY.rst /igel/HISTORY.rst +COPY setup.py /igel/setup.py +RUN cd /igel && python setup.py install + +VOLUME /data +WORKDIR /data + +ENTRYPOINT ["igel"] +CMD ["igel"] diff --git a/igel/HISTORY.rst b/igel/HISTORY.rst new file mode 100644 index 0000000..1399662 --- /dev/null +++ b/igel/HISTORY.rst @@ -0,0 +1,147 @@ +======= +History +======= + +0.4.0 (2021-06-22) +------------------- + +* Added a possibility to serve a trained model using FastAPI +* Updated cli +* Fixed bugs +* Added more unit tests +* migrated to gh actions +* migrated to poetry + + +0.3.1 (2020-10-31) +------------------- + +* Added support for multiple data types (excel, json and html) +* Added igel gui as a command +* Integrated igel UI + +0.3.0 (2020-10-15) +------------------- + +* Provided a way to use reproducible results +* Added support for random state generation +* Fixed bug in dataset options +* Linked Igel-UI + +0.2.9 (2020-10-12) +------------------- + +* Fixed bug in clustering +* added a clustering example using kmeans +* added support for clustering arguments + +0.2.8 (2020-10-09) +------------------- + +* implemented hyperparameter search +* added a hyperparameter example + + +0.2.7 (2020-10-05) +------------------- + +* removed colorama (since it was causing bugs on jupyter) +* improved interactive mode +* added commands to get version and infos about the package + +0.2.6 (2020-10-04) +------------------- + +* added interactive mode in cli +* added colors in cli to improve readability +* updated setup + +0.2.5 (2020-10-03) +------------------- + +* fixed igel initialization step +* updated examples +* added gids to readme + +0.2.4 (2020-09-28) +------------------- + +* added support for json as a configuration file +* added support for providing read data options +* added CV class support +* added cross validation support + +0.2.3 (2020-09-26) +------------------- + +* added clustering support +* all clustering classes of sklearn are now supported + +0.2.2 (2020-09-23) +------------------- + +* added init command +* users can get started quickly by getting a default yaml file "on the fly" + +0.2.1 (2020-09-21) +------------------- + +* added support for other ensemble models like adaboost, extra trees etc.. + + +0.2.0 (2020-09-19) +------------------- + +* added the experiment command +* fixed bugs in predict +* fixed sphinx docs dependencies + +0.1.9 (2020-09-18) +------------------- + +* added support for multioutput regression and classification +* fixed bug in preparing predict data +* provided a way to evaluate models +* changed class from IgelModel to Igel +* updated igel cli + +0.1.8 (2020-09-13) +------------------ +* fixed predict function bugs and added examples + +0.1.7 (2020-09-12) +------------------ +* implemented optional arguments in sklearn models + + +0.1.5 (2020-09-10) +------------------ +* implemented encoding and scaling methods + +0.1.4 (2020-09-08) +------------------ +* support for all sklearn models + +0.1.3 (2020-09-07) +------------------ +* implemented basic dataset operations + +0.1.0 (2020-09-05) +------------------ +* stable release with an end to end pipeline + +0.0.6 (2020-09-01) +------------------ +* Added validation on arguments and provided an example + +0.0.5 (2020-08-31) +------------------ +* Added logging and changed file keyword to yaml_file + +0.0.3 (2020-08-30) +------------------ +* First functional package + +0.0.1 (2020-08-27) +------------------ +* First release on PyPI. diff --git a/igel/KNOWN_ISSUES.md b/igel/KNOWN_ISSUES.md new file mode 100644 index 0000000..474b24f --- /dev/null +++ b/igel/KNOWN_ISSUES.md @@ -0,0 +1,14 @@ +# Known Issues + +This document lists known limitations and issues in the project. + +## Current Known Issues + +- [ ] Custom model architectures are experimental and may not support all scikit-learn features. +- [ ] Some advanced hyperparameter search options may not be available for all models. +- [ ] Windows users may encounter PowerShell script execution policy errors when activating virtual environments (see README for workaround). +- [ ] Please check open [GitHub Issues](https://github.com/HeerakKashyap/igel/issues) for the latest updates. + +--- + +If you encounter a new issue, please report it on [GitHub Issues](https://github.com/HeerakKashyap/igel/issues). diff --git a/igel/LICENSE b/igel/LICENSE new file mode 100644 index 0000000..c9abe7c --- /dev/null +++ b/igel/LICENSE @@ -0,0 +1,23 @@ +MIT License + +Copyright (c) 2020, Nidhal Baccouri + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + + diff --git a/igel/MANIFEST.in b/igel/MANIFEST.in new file mode 100644 index 0000000..73f3032 --- /dev/null +++ b/igel/MANIFEST.in @@ -0,0 +1,12 @@ +include AUTHORS.rst +include CONTRIBUTING.rst +include HISTORY.rst +include LICENSE +include docs/README.rst + +recursive-include tests * +recursive-exclude * __pycache__ +recursive-exclude * *.py[co] + + +recursive-include docs *.rst conf.py Makefile make.bat *.jpg *.png *.gif diff --git a/igel/Makefile b/igel/Makefile new file mode 100644 index 0000000..42fdda7 --- /dev/null +++ b/igel/Makefile @@ -0,0 +1,173 @@ +SHELL := /usr/bin/env bash + +IMAGE := igel +VERSION := latest + +# ----------------------------------------------------------------------------- +# The following section sets command flags for various tools (poetry, pip, safety, etc.) +# based on whether "strict" mode is enabled. This is done by setting each flag variable +# to either "-" (non-strict) or "" (strict). +# +# Why this is done this way: +# Make does not support creating variables in a loop or dynamically generating variable +# names in a portable way. While GNU Make has some advanced features, this Makefile aims +# to be as portable and readable as possible. Therefore, each flag variable is set +# individually, even though this results in repetitive code. +# +# If you know a more concise, portable way to achieve this, contributions are welcome! +# ----------------------------------------------------------------------------- +#! An ugly hack to create individual flags +ifeq ($(STRICT), 1) + POETRY_COMMAND_FLAG = + PIP_COMMAND_FLAG = + SAFETY_COMMAND_FLAG = + BANDIT_COMMAND_FLAG = + SECRETS_COMMAND_FLAG = + BLACK_COMMAND_FLAG = + DARGLINT_COMMAND_FLAG = + ISORT_COMMAND_FLAG = + MYPY_COMMAND_FLAG = +else + POETRY_COMMAND_FLAG = - + PIP_COMMAND_FLAG = - + SAFETY_COMMAND_FLAG = - + BANDIT_COMMAND_FLAG = - + SECRETS_COMMAND_FLAG = - + BLACK_COMMAND_FLAG = - + DARGLINT_COMMAND_FLAG = - + ISORT_COMMAND_FLAG = - + MYPY_COMMAND_FLAG = - +endif + +##! Please tell me how to use `for loops` to create variables in Makefile :( +##! If you have better idea, please PR me in https://github.com/TezRomacH/python-package-template + +ifeq ($(POETRY_STRICT), 1) + POETRY_COMMAND_FLAG = +else ifeq ($(POETRY_STRICT), 0) + POETRY_COMMAND_FLAG = - +endif + +ifeq ($(PIP_STRICT), 1) + PIP_COMMAND_FLAG = +else ifeq ($(PIP_STRICT), 0) + PIP_COMMAND_FLAG = - +endif + +ifeq ($(SAFETY_STRICT), 1) + SAFETY_COMMAND_FLAG = +else ifeq ($(SAFETY_STRICT), 0) + SAFETY_COMMAND_FLAG = - +endif + +ifeq ($(BANDIT_STRICT), 1) + BANDIT_COMMAND_FLAG = +else ifeq ($(BANDIT_STRICT), 0) + BANDIT_COMMAND_FLAG = - +endif + +ifeq ($(SECRETS_STRICT), 1) + SECRETS_COMMAND_FLAG = +else ifeq ($(SECRETS_STRICT), 0) + SECRETS_COMMAND_FLAG = - +endif + +ifeq ($(BLACK_STRICT), 1) + BLACK_COMMAND_FLAG = +else ifeq ($(BLACK_STRICT), 0) + BLACK_COMMAND_FLAG = - +endif + +ifeq ($(DARGLINT_STRICT), 1) + DARGLINT_COMMAND_FLAG = +else ifeq ($(DARGLINT_STRICT), 0) + DARGLINT_COMMAND_FLAG = - +endif + +ifeq ($(ISORT_STRICT), 1) + ISORT_COMMAND_FLAG = +else ifeq ($(ISORT_STRICT), 0) + ISORT_COMMAND_FLAG = - +endif + + +ifeq ($(MYPY_STRICT), 1) + MYPY_COMMAND_FLAG = +else ifeq ($(MYPY_STRICT), 0) + MYPY_COMMAND_FLAG = - +endif + +#! The end of the ugly part. I'm really sorry + +.PHONY: download-poetry +download-poetry: + curl -sSL https://install.python-poetry.org | python3 - + +.PHONY: install +install: + poetry lock -n + poetry install -n +ifneq ($(NO_PRE_COMMIT), 1) + poetry run pre-commit install +endif +# +#.PHONY: check-safety +#check-safety: +# $(POETRY_COMMAND_FLAG)poetry check +# $(PIP_COMMAND_FLAG)poetry run pip check +# $(SAFETY_COMMAND_FLAG)poetry run safety check +# $(BANDIT_COMMAND_FLAG)poetry run bandit -ll -r igel/ + +.PHONY: check-style +check-style: + $(BLACK_COMMAND_FLAG)poetry run black --config pyproject.toml --diff --check ./ + $(DARGLINT_COMMAND_FLAG)poetry run darglint -v 2 **/*.py + $(ISORT_COMMAND_FLAG)poetry run isort --settings-path pyproject.toml --check-only **/*.py + $(MYPY_COMMAND_FLAG)poetry run mypy --config-file setup.cfg igel tests/**/*.py + +.PHONY: codestyle +codestyle: + -poetry run pyupgrade --py37-plus **/*.py + poetry run isort --settings-path pyproject.toml **/*.py + poetry run black --config pyproject.toml ./ + +.PHONY: test +test: + cd tests/test_igel/ && poetry run pytest + +.PHONY: lint +lint: test check-safety check-style + +# Example: make docker VERSION=latest +# Example: make docker IMAGE=some_name VERSION=0.1.0 +.PHONY: docker +docker: + @echo Building docker $(IMAGE):$(VERSION) ... + docker build \ + -t $(IMAGE):$(VERSION) . \ + -f ./docker/Dockerfile --no-cache + +# Example: make clean_docker VERSION=latest +# Example: make clean_docker IMAGE=some_name VERSION=0.1.0 +.PHONY: clean_docker +clean_docker: + @echo Removing docker $(IMAGE):$(VERSION) ... + docker rmi -f $(IMAGE):$(VERSION) + +.PHONY: clean_build +clean: + rm -rf build/ + +.PHONY: clean +clean: clean_build clean_docker + + +git: ## add, commit and push in one command + git add . + git commit -m "$m" + git push origin master + +pushDev: ## push to the dev branch + git add . + git commit -m "$m" + git push origin develop diff --git a/igel/assets/igel-eval.gif b/igel/assets/igel-eval.gif new file mode 100644 index 0000000..9f77c21 Binary files /dev/null and b/igel/assets/igel-eval.gif differ diff --git a/igel/assets/igel-experiment.gif b/igel/assets/igel-experiment.gif new file mode 100644 index 0000000..bdd4015 Binary files /dev/null and b/igel/assets/igel-experiment.gif differ diff --git a/igel/assets/igel-fit-interactive.gif b/igel/assets/igel-fit-interactive.gif new file mode 100644 index 0000000..a97ec93 Binary files /dev/null and b/igel/assets/igel-fit-interactive.gif differ diff --git a/igel/assets/igel-fit.gif b/igel/assets/igel-fit.gif new file mode 100644 index 0000000..af2e4a7 Binary files /dev/null and b/igel/assets/igel-fit.gif differ diff --git a/igel/assets/igel-help.gif b/igel/assets/igel-help.gif new file mode 100644 index 0000000..b648e45 Binary files /dev/null and b/igel/assets/igel-help.gif differ diff --git a/igel/assets/igel-init-interactive.gif b/igel/assets/igel-init-interactive.gif new file mode 100644 index 0000000..94e742c Binary files /dev/null and b/igel/assets/igel-init-interactive.gif differ diff --git a/igel/assets/igel-init.gif b/igel/assets/igel-init.gif new file mode 100644 index 0000000..1e62d5c Binary files /dev/null and b/igel/assets/igel-init.gif differ diff --git a/igel/assets/igel-pred.gif b/igel/assets/igel-pred.gif new file mode 100644 index 0000000..8068864 Binary files /dev/null and b/igel/assets/igel-pred.gif differ diff --git a/igel/assets/igel-predict.gif b/igel/assets/igel-predict.gif new file mode 100644 index 0000000..5048fba Binary files /dev/null and b/igel/assets/igel-predict.gif differ diff --git a/igel/assets/logo.jpg b/igel/assets/logo.jpg new file mode 100644 index 0000000..a8ea467 Binary files /dev/null and b/igel/assets/logo.jpg differ diff --git a/igel/deprecated/old_cli.py b/igel/deprecated/old_cli.py new file mode 100644 index 0000000..c18f7dd --- /dev/null +++ b/igel/deprecated/old_cli.py @@ -0,0 +1,664 @@ +"""Console script for igel.""" +import argparse +import logging +import os +import subprocess +import sys +from pathlib import Path + +import igel +import pandas as pd +from igel import Igel, metrics_dict, models_dict +from igel.constants import Constants +from igel.servers import fastapi_server + +logger = logging.getLogger(__name__) + + +class CLI: + """CLI describes a command line interface for interacting with igel, there + are several different functions that can be performed. + + """ + + available_args = { + # fit, evaluate and predict args: + "dp": "data_path", + "yml": "yaml_path", + "DP": "data_paths", + "res_dir": "model_results_dir", + # models arguments + "name": "model_name", + "model": "model_name", + "type": "model_type", + "tg": "target", + # host and port for serving the model + "h": "host", + "p": "port", + } + + def __init__(self): + self.parser = argparse.ArgumentParser( + description="Igel CLI Runner", + usage=f""" + ___ _ ____ _ ___ +|_ _|__ _ ___| | / ___| | |_ _| + | |/ _` |/ _ \\ | | | | | | | + | | (_| | __/ | | |___| |___ | | +|___\\__, |\\___|_| \\____|_____|___| + |___/ + + +igel [] + +- Available commands: + init initialize a yaml file with default parameters + fit fits a model + evaluate evaluate the performance of a pre-fitted model + predict predicts using a pre-fitted model + experiment this command will run fit, evaluate and predict together + serve serve the trained model + gui open the igel UI (make sure you have nodejs installed) + help get help about how to use igel + info get info & metadata about igel + version get the version of igel installed on your machine + models get a list of supported machine learning algorithms/models + metrics get a list of all supported metrics + +- Available arguments: + + # for usage with the fit, evaluate or predict command: + -dp Path to your dataset (dp stand for data_path, you can use --data_path instead) + -yml Path to your yaml file (you can use --yaml_path instead) + + # for usage with the experiment command: + -DP Paths to data you want to use for fitting, + evaluating and predict respectively. (you can use --data_paths instead) + + -yml Path to the yaml file that will be used when fitting the model. + + # for usage with the serve command: + -res_dir Path to the model_results directory generated by igel after training the model + + # for getting help with the models command: + -type type of the model you want to get help on + -> whether regression, classification or clustering. (you can use --model_type instead) + + -name name of the model you want to get help on. (you can use --model_name instead) + + +Note: you can run the commands without providing additional arguments, which will run interactive mode, where + you will be prompted to enter your arguments on the fly. + +--------------------------------------------------------------------------------------------------------------- + +- HowTo: + + - you can always type igel -h to print the help + - you can run igel version to get the version installed + - you can run igel info to get meta data about the package + + ----------------------------------------------------------- + + you can let igel generate a boilerplate config file for you by running "igel init". This will automatically + create a yaml file in the working directory with some default parameters to get you started fast + + - example for RandomForest regressor: igel init -type regression -model RandomForest + - you can also run this in interactive mode by just running: igel init + + ----------------------------------------------------------- + + you can get help on supported models by running igel models in your terminal. this will list all supported + models in a table. Additionally, you will be prompted to enter a model name and type that you want to get + help about. You can also pass arguments when running the command. + + - example for getting help on how to use RandomForest: igel models -type regression -name RandomForest + + ------------------------------------------------------------ + + you can also get help on supported metrics. Just run igel metrics to get all supported metrics + + ------------------------------------------------------------ + + Training/fitting a model is very easy in igel. You can just run igel fit to enter interactive mode, where + you will be prompted to enter path to your dataset and config file. You can also provide the path to + your dataset and config file directly if you want by running: + + - example: igel fit -dp "path_to_data" -yml "path_to_yaml_file" + + This will fit a model and save it in a folder called model_results in your current working directory + + ------------------------------------------------------------- + + Evaluating a model is also very easy. Just run the igel evaluate command to enter interactive mode. + Otherwise you can always enter the arguments directly. + + - example: igel evaluate -dp "path_to_data" + + This will evaluate the pre-trained model and save results in an evaluation.json file in the model_results dir. + + -------------------------------------------------------------- + + Using the pre-trained model to generate predictions is straightforward. Just run the igel predict command, + which will run interactive mode, where you will be prompted to enter path to your predict data. Same as + other commands, you can also provide arguments directly when running this: + + - example: igel predict -dp "path_to_data" + + This will generate predictions and save it in a predictions.csv file in the model_results dir. + + -------------------------------------------------------------- + + you can be lazy and run the fit, evaluate and predict command in one simple command called experiment. + Same as other command, just run igel experiment to enter interactive mode or provide arguments directly. + + + - example: igel experiment -DP "path_to_train_data \\ + path_to_evaluation_data \\ + path_to_data_you_want_to_predict_on" -yml "path_to_yaml_file" + + This will run the fit command using the train data, then evaluate your model using the evaluation data + and finally generate predictions on the predict data. + + ---------------------------------------------------------------- + + The next step is to use your model in production. Igel helps you with this task too by providing the serve command. + Running the serve command will tell igel to serve your model. Precisely, igel will automatically build + a REST server and serve your model on a specific host and port, which you can configure by passing these as + cli options. + + ---------------------------------------------------------------- + + Finally, if you are a non-technical user and don't want to use the terminal, then consider using the igel UI. + You can run this from igel by typing igel gui, which will run the igel UI application, where you can use igel + with a few clicks. Make sure you have nodejs installed for this. Check the official repo for more infos. + + ---------------------------------------------------------------- + + + """, + ) + + self.parser.add_argument("command", help="Subcommand to run") + self.cmd = self.parse_command() + self.args = sys.argv[2:] + self.dict_args = self.convert_args_to_dict() + getattr(self, self.cmd.command)() + + def validate_args(self, dict_args: dict) -> dict: + """ + validate arguments entered by the user and transform short args to the representation needed by igel + @param dict_args: dict of arguments + @return: new validated and transformed args + + """ + d_args = {} + for k, v in dict_args.items(): + if ( + k not in self.available_args.keys() + and k not in self.available_args.values() + ): + logger.warning(f"Unrecognized argument -> {k}") + self.parser.print_help() + exit(1) + + elif k in self.available_args.values(): + d_args[k] = v + + else: + d_args[self.available_args[k]] = v + + return d_args + + def convert_args_to_dict(self) -> dict: + """ + convert args list to a dictionary + @return: args as dictionary + """ + + dict_args = { + self.args[i].replace("-", ""): self.args[i + 1] + for i in range(0, len(self.args) - 1, 2) + } + dict_args = self.validate_args(dict_args) + dict_args["cmd"] = self.cmd.command + return dict_args + + def parse_command(self): + """ + parse command, which represents the function that will be called by igel + @return: command entered by the user + """ + # parse_args defaults to [1:] for args, but you need to + # exclude the rest of the args too, or validation will fail + cmd = self.parser.parse_args(sys.argv[1:2]) + if not hasattr(self, cmd.command): + logger.warning("Unrecognized command") + self.parser.print_help() + exit(1) + # use dispatch pattern to invoke method with same name + return cmd + + def help(self, *args, **kwargs): + self.parser.print_help() + + def gui(self, *args, **kwargs): + igel_ui_path = Path(os.getcwd()) / "igel-ui" + if not Path.exists(igel_ui_path): + subprocess.check_call( + ["git"] + ["clone", "https://github.com/nidhaloff/igel-ui.git"] + ) + logger.info(f"igel UI cloned successfully") + + os.chdir(igel_ui_path) + logger.info(f"switching to -> {igel_ui_path}") + logger.info(f"current dir: {os.getcwd()}") + logger.info(f"make sure you have nodejs installed!!") + + subprocess.Popen(["node", "npm", "install", "open"], shell=True) + subprocess.Popen( + ["node", "npm", "install electron", "open"], shell=True + ) + logger.info("installing dependencies ...") + logger.info(f"dependencies installed successfully") + logger.info(f"node version:") + subprocess.check_call("node -v", shell=True) + logger.info(f"npm version:") + subprocess.check_call("npm -v", shell=True) + subprocess.check_call("npm i electron", shell=True) + logger.info("running igel UI...") + subprocess.check_call("npm start", shell=True) + + def init(self, *args, **kwargs): + """ + initialize a dummy/default yaml file as a starting point. The user can provide args directly in the terminal + usage: + igel init + + if not args are provided, the user will be prompted to enter basic information. + """ + d = dict(self.dict_args) + d.pop("cmd") + if not d: + print( + f"" + f"{'*' * 10} You entered interactive mode! {'*' * 10} \n" + f"This is happening because you didn't enter all mandatory arguments in order to use the cli\n" + f"Therefore, you will need to provide few information before proceeding.\n" + ) + model_type = ( + input( + f"enter type of the problem you want to solve: [regression] " + ) + or "regression" + ) + d["model_type"] = model_type + model_name = ( + input( + f"enter algorithm you want to use: [NeuralNetwork] " + ) + or "NeuralNetwork" + ) + d["model_name"] = model_name + target = input( + f"enter the target you want to predict " + "(this is usually a column name in your csv dataset): " + ) + d["target"] = target + + Igel.create_init_mock_file(**d) + + def _accept_user_input( + self, + yaml_needed: bool = False, + default_data_path: str = "./train_data.csv", + default_yaml_path: str = "./igel.yaml", + ): + """ + accept user input if the user did not provide all mandatory args in the terminal. + """ + print( + f"" + f"{'*' * 10} You entered interactive mode! {'*' * 10} \n" + f"This is happening because you didn't enter all mandatory arguments in order to use the cli\n" + f"Therefore, you will need to provide few information before proceeding.\n" + ) + data_path = ( + input(f"enter path to your data: [{default_data_path}] ") + or default_data_path + ) + self.dict_args["data_path"] = data_path + if yaml_needed: + yaml_path = ( + input( + f"enter path to your yaml file: [{default_yaml_path}] " + ) + or default_yaml_path + ) + self.dict_args["yaml_path"] = yaml_path + + def fit(self, *args, **kwargs): + print( + r""" + _____ _ _ + |_ _| __ __ _(_)_ __ (_)_ __ __ _ + | || '__/ _` | | '_ \| | '_ \ / _` | + | || | | (_| | | | | | | | | | (_| | + |_||_| \__,_|_|_| |_|_|_| |_|\__, | + |___/ + """ + ) + d = dict(self.dict_args) + d.pop("cmd") + if not d: + self._accept_user_input(yaml_needed=True) + + Igel(**self.dict_args) + + def predict(self, *args, **kwargs): + print( + """ + ____ _ _ _ _ + | _ \\ _ __ ___ __| (_) ___| |_(_) ___ _ __ + | |_) | '__/ _ \\/ _` | |/ __| __| |/ _ \\| '_ \ + | __/| | | __/ (_| | | (__| |_| | (_) | | | | + |_| |_| \\___|\\__,_|_|\\___|\\__|_|\\___/|_| |_| + + + """ + ) + d = dict(self.dict_args) + d.pop("cmd") + if not d: + self._accept_user_input() + Igel(**self.dict_args) + + def evaluate(self, *args, **kwargs): + print( + """ + _____ _ _ _ + | ____|_ ____ _| |_ _ __ _| |_(_) ___ _ __ + | _| \\ \\ / / _` | | | | |/ _` | __| |/ _ \\| '_ \ + | |___ \\ V / (_| | | |_| | (_| | |_| | (_) | | | | + |_____| \\_/ \\__,_|_|\\__,_|\\__,_|\\__|_|\\___/|_| |_| + + """ + ) + d = dict(self.dict_args) + d.pop("cmd") + if not d: + self._accept_user_input() + + Igel(**self.dict_args) + + def _print_models_overview(self): + print(f"\nIgel's supported models overview: \n") + reg_algs = list(models_dict.get("regression").keys()) + clf_algs = list(models_dict.get("classification").keys()) + cluster_algs = list(models_dict.get("clustering").keys()) + df_algs = ( + pd.DataFrame.from_dict( + { + "regression": reg_algs, + "classification": clf_algs, + "clustering": cluster_algs, + }, + orient="index", + ) + .transpose() + .fillna("----") + ) + + df = self._tableize(df_algs) + print(df) + + def _show_model_infos(self, model_name: str, model_type: str): + if not model_name: + print(f"Please enter a supported model") + self._print_models_overview() + else: + if not model_type: + print( + f"Please enter a type argument to get help on the chosen model\n" + f"type can be whether regression, classification or clustering \n" + ) + self._print_models_overview() + return + if model_type not in ("regression", "classification", "clustering"): + raise Exception( + f"{model_type} is not supported! \n" + f"model_type need to be regression, classification or clustering" + ) + + models = models_dict.get(model_type) + model_data = models.get(model_name) + model, link, *cv_class = model_data.values() + print( + f"model type: {model_type} \n" + f"model name: {model_name} \n" + f"sklearn model class: {model.__name__} \n" + f"{'-' * 60}\n" + f"You can click the link below to know more about the optional arguments\n" + f"that you can use with your chosen model ({model_name}).\n" + f"You can provide these optional arguments in the yaml file if you want to use them.\n" + f"link:\n{link} \n" + ) + + def models(self): + """ + show an overview of all models supported by igel + """ + if not self.dict_args or len(self.dict_args.keys()) <= 1: + self._print_models_overview() + print("-" * 100) + model_name = input( + "Enter the model name, you want to get infos about (e.g NeuralNetwork): " + ) + model_type = input( + "Enter the type (choose from regression, classification or clustering): " + ) + if model_name and model_type: + self._show_model_infos(model_name, model_type) + else: + model_name = self.dict_args.get("model_name", None) + model_type = self.dict_args.get("model_type", None) + self._show_model_infos(model_name, model_type) + + def metrics(self): + """ + show an overview of all metrics supported by igel + """ + print(f"\nIgel's supported metrics overview: \n") + reg_metrics = [func.__name__ for func in metrics_dict.get("regression")] + clf_metrics = [ + func.__name__ for func in metrics_dict.get("classification") + ] + + df_metrics = ( + pd.DataFrame.from_dict( + {"regression": reg_metrics, "classification": clf_metrics}, + orient="index", + ) + .transpose() + .fillna("----") + ) + + df_metrics = self._tableize(df_metrics) + print(df_metrics) + + def experiment(self): + """ + run a whole experiment: this is a pipeline that includes fit, evaluate and predict. + """ + print( + r""" + _____ _ _ + | ____|_ ___ __ ___ _ __(_)_ __ ___ ___ _ __ | |_ + | _| \ \/ / '_ \ / _ \ '__| | '_ ` _ \ / _ \ '_ \| __| + | |___ > <| |_) | __/ | | | | | | | | __/ | | | |_ + |_____/_/\_\ .__/ \___|_| |_|_| |_| |_|\___|_| |_|\__| + |_| + + """ + ) + d = dict(self.dict_args) + d.pop("cmd") + if not d: + default_train_data_path = "./train_data.csv" + default_eval_data_path = "./eval_data.csv" + default_test_data_path = "./test_data.csv" + default_yaml_path = "./igel.yaml" + print( + f"" + f"{'*' * 10} You entered interactive mode! {'*' * 10} \n" + f"This is happening because you didn't enter all mandatory arguments in order to use the cli\n" + f"Therefore, you will need to provide few information before proceeding.\n" + ) + train_data_path = ( + input( + f"enter path to your data: [{default_train_data_path}] " + ) + or default_train_data_path + ) + eval_data_path = ( + input( + f"enter path to your data: [{default_eval_data_path}] " + ) + or default_eval_data_path + ) + test_data_path = ( + input( + f"enter path to your data: [{default_test_data_path}] " + ) + or default_test_data_path + ) + yaml_path = ( + input( + f"enter path to your yaml file: [{default_yaml_path}] " + ) + or default_yaml_path + ) + + # prepare the dict arguments: + train_args = { + "cmd": "fit", + "yaml_path": yaml_path, + "data_path": train_data_path, + } + eval_args = {"cmd": "evaluate", "data_path": eval_data_path} + pred_args = {"cmd": "predict", "data_path": test_data_path} + + else: + data_paths = self.dict_args["data_paths"] + yaml_path = self.dict_args["yaml_path"] + ( + train_data_path, + eval_data_path, + pred_data_path, + ) = data_paths.strip().split(" ") + # print(f"{train_data_path} | {eval_data_path} | {test_data_path}") + train_args = { + "cmd": "fit", + "yaml_path": yaml_path, + "data_path": train_data_path, + } + eval_args = {"cmd": "evaluate", "data_path": eval_data_path} + pred_args = {"cmd": "predict", "data_path": pred_data_path} + + Igel(**train_args) + Igel(**eval_args) + Igel(**pred_args) + + def serve(self): + """ + expose a REST endpoint in order to use the trained ML model + """ + try: + os.environ[Constants.model_results_path] = self.dict_args[ + "model_results_dir" + ] + uvicorn_params = {} + if "host" in self.dict_args and "port" in self.dict_args: + uvicorn_params["host"] = self.dict_args.get("host") + uvicorn_params["port"] = int(self.dict_args.get("port")) + + fastapi_server.run(**uvicorn_params) + + except Exception as ex: + logger.exception(ex) + + def _tableize(self, df): + """ + pretty-print a dataframe as table + """ + if not isinstance(df, pd.DataFrame): + return + df_columns = df.columns.tolist() + max_len_in_lst = lambda lst: len(sorted(lst, reverse=True, key=len)[0]) + align_center = ( + lambda st, sz: "{0}{1}{0}".format( + " " * (1 + (sz - len(st)) // 2), st + )[:sz] + if len(st) < sz + else st + ) + align_right = ( + lambda st, sz: "{}{} ".format(" " * (sz - len(st) - 1), st) + if len(st) < sz + else st + ) + max_col_len = max_len_in_lst(df_columns) + max_val_len_for_col = { + col: max_len_in_lst(df.iloc[:, idx].astype("str")) + for idx, col in enumerate(df_columns) + } + col_sizes = { + col: 2 + max(max_val_len_for_col.get(col, 0), max_col_len) + for col in df_columns + } + build_hline = lambda row: "+".join( + ["-" * col_sizes[col] for col in row] + ).join(["+", "+"]) + build_data = lambda row, align: "|".join( + [ + align(str(val), col_sizes[df_columns[idx]]) + for idx, val in enumerate(row) + ] + ).join(["|", "|"]) + hline = build_hline(df_columns) + out = [hline, build_data(df_columns, align_center), hline] + for _, row in df.iterrows(): + out.append(build_data(row.tolist(), align_right)) + out.append(hline) + return "\n".join(out) + + def version(self): + print(f"igel version: {igel.__version__}") + + def info(self): + print( + f""" + package name: igel + version: {igel.__version__} + author: Nidhal Baccouri + maintainer: Nidhal Baccouri + contact: nidhalbacc@gmail.com + license: MIT + description: use machine learning without writing code + dependencies: pandas, sklearn, pyyaml + requires python: >= 3.6 + First release: 27.08.2020 + official repo: https://github.com/nidhaloff/igel + written in: 100% python + status: stable + operating system: independent + """ + ) + + +def main(): + CLI() + + +if __name__ == "__main__": + main() diff --git a/igel/docs/Makefile b/igel/docs/Makefile new file mode 100644 index 0000000..3fb266a --- /dev/null +++ b/igel/docs/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXOPTS = +SPHINXBUILD = python -msphinx +SPHINXPROJ = igel +SOURCEDIR = . +BUILDDIR = _build + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/igel/docs/README.rst b/igel/docs/README.rst new file mode 100644 index 0000000..bc627f8 --- /dev/null +++ b/igel/docs/README.rst @@ -0,0 +1,1014 @@ +.. image:: https://img.shields.io/badge/code%20style-black-000000.svg + :alt: Code style: black + +==== +igel +==== + +| + +.. image:: https://img.shields.io/pypi/v/igel?color=green + :alt: PyPI + :target: https://pypi.python.org/pypi/igel +.. image:: https://img.shields.io/github/workflow/status/nidhaloff/igel/build + :target: https://github.com/nidhaloff/igel/actions/workflows/build.yml + :alt: GitHub Workflow Status +.. image:: https://pepy.tech/badge/igel + :target: https://pepy.tech/project/igel +.. image:: https://readthedocs.org/projects/igel/badge/?version=latest + :target: https://igel.readthedocs.io/en/latest/?badge=latest + :alt: Documentation Status + +.. image:: https://img.shields.io/pypi/wheel/igel + :alt: PyPI - Wheel + :target: https://pypi.python.org/pypi/igel + + +.. image:: https://img.shields.io/github/stars/nidhaloff/igel?style=social + :alt: GitHub Repo stars + :target: https://pypi.python.org/pypi/igel + +.. image:: https://img.shields.io/twitter/url?url=https%3A%2F%2Ftwitter.com%2FNidhalBaccouri + :alt: Twitter URL + :target: https://twitter.com/NidhalBaccouri + +| + +# Quickstart + +Install igel using pip: + +```bash +pip install igel +``` + +Train a model with a config file: + +```bash +igel fit --data_path=path/to/data.csv --yaml_path=path/to/config.yaml +``` + +For more details and advanced usage, see the sections below or the [documentation](link-to-docs). + +A delightful machine learning tool that allows you to train/fit, test and use models **without writing code** + +.. note:: + I'm also working on a GUI desktop app for igel based on people's requests. You can find it under + `Igel-UI `_. + +* Free software: MIT license +* Documentation: https://igel.readthedocs.io. + +.. contents:: Table of Contents + :depth: 3 + +| +| + +Introduction +------------------ + +The goal of the project is to provide machine learning for **everyone**, both technical and non-technical +users. + +I needed a tool sometimes, which I can use to fast create a machine learning prototype. Whether to build +some proof of concept, create a fast draft model to prove a point or use auto ML. I find myself often stuck at writing +boilerplate code and thinking too much where to start. Therefore, I decided to create this tool. + +igel is built on top of other ML frameworks. It provides a simple way to use machine learning without writing +a **single line of code**. Igel is **highly customizable**, but only if you want to. Igel does not force you to +customize anything. Besides default values, igel can use auto-ml features to figure out a model that can work great with your data. + +All you need is a **yaml** (or **json**) file, where you need to describe what you are trying to do. That's it! + +Igel supports regression, classification, clustering, and few-shot learning. +Igel's supports auto-ml features like ImageClassification and TextClassification + +Igel supports most used dataset types in the data science field. For instance, your input dataset can be +a csv, txt, excel sheet, json or even html file that you want to fetch. If you are using auto-ml features, then you can even +feed raw data to igel and it will figure out how to deal with it. More on this later in the examples. + + +Features +--------- + +- Supports most dataset types (csv, txt, excel, json, html) even just raw data stored in folders +- Supports all state of the art machine learning models (even preview models) +- Supports different data preprocessing methods +- Provides flexibility and data control while writing configurations +- Supports cross validation +- Supports both hyperparameter search (version >= 0.2.8) +- Supports yaml and json format +- Usage from GUI +- Supports different sklearn metrics for regression, classification and clustering +- Supports multi-output/multi-target regression and classification +- Supports multi-processing for parallel model construction +- Support for **auto machine learning** +- **Few-shot learning** with MAML and Prototypical Networks +- **Domain adaptation** and transfer learning utilities + +Installation +------------- + +- The easiest way is to install igel using `pip `_ + +.. code-block:: console + + $ pip install -U igel + +Models +------- + +Igel's supported models: + +.. code-block:: console + + +--------------------+----------------------------+-------------------------+-------------------------+ + | regression | classification | clustering | few_shot_learning | + +--------------------+----------------------------+-------------------------+-------------------------+ + | LinearRegression | LogisticRegression | KMeans | + | Lasso | Ridge | AffinityPropagation | + | LassoLars | DecisionTree | Birch | + | BayesianRegression | ExtraTree | AgglomerativeClustering | + | HuberRegression | RandomForest | FeatureAgglomeration | + | Ridge | ExtraTrees | DBSCAN | + | PoissonRegression | SVM | MiniBatchKMeans | + | ARDRegression | LinearSVM | SpectralBiclustering | + | TweedieRegression | NuSVM | SpectralCoclustering | + | TheilSenRegression | NearestNeighbor | SpectralClustering | + | GammaRegression | NeuralNetwork | MeanShift | + | RANSACRegression | PassiveAgressiveClassifier | OPTICS | + | DecisionTree | Perceptron | KMedoids | + | ExtraTree | BernoulliRBM | ---- | + | RandomForest | BoltzmannMachine | ---- | + | ExtraTrees | CalibratedClassifier | ---- | + | SVM | Adaboost | ---- | + | LinearSVM | Bagging | ---- | + | NuSVM | GradientBoosting | ---- | + | NearestNeighbor | BernoulliNaiveBayes | ---- | + | NeuralNetwork | CategoricalNaiveBayes | ---- | + | ElasticNet | ComplementNaiveBayes | ---- | + | BernoulliRBM | GaussianNaiveBayes | ---- | + | BoltzmannMachine | MultinomialNaiveBayes | ---- | + | Adaboost | ---- | ---- | + | Bagging | ---- | ---- | + | GradientBoosting | ---- | ---- | ---- | + | ---- | ---- | ---- | MAML | + | ---- | ---- | ---- | PrototypicalNetwork | + +--------------------+----------------------------+-------------------------+-------------------------+ + +For auto ML: + +- ImageClassifier +- TextClassifier +- ImageRegressor +- TextRegressor +- StructeredDataClassifier +- StructeredDataRegressor +- AutoModel + +Quick Start +------------ + +The help command is very useful to check supported commands and corresponding args/options + +.. code-block:: console + + $ igel --help + +You can also run help on sub-commands, for example: + +.. code-block:: console + + $ igel fit --help + + +Igel is highly customizable. If you know what you want and want to configure your model manually, +then check the next sections, which will guide you on how to write a yaml or a json config file. +After that, you just have to tell igel, what to do and where to find your data and config file. +Here is an example: + +.. code-block:: console + + $ igel fit --data_path 'path_to_your_csv_dataset.csv' --yaml_path 'path_to_your_yaml_file.yaml' + +However, you can also use the auto-ml features and let igel do everything for you. +A great example for this would be image classification. Let's imagine you already have a dataset +of raw images stored in a folder called **images** + +All you have to do is run: + +.. code-block:: console + + $ igel auto-train --data_path 'path_to_your_images_folder' --task ImageClassification + +That's it! Igel will read the images from the directory, +process the dataset (converting to matrices, rescale, split, etc...) and start training/optimizing +a model that works good on your data. As you can see it's pretty easy, you just have to provide the path +to your data and the task you want to perform. + +.. note:: + + This feature is computationally expensive as igel would try many + different models and compare their performance in order to find the 'best' one. + + + +Usage +------ + +You can run the help command to get instructions. You can also run help on sub-commands! + +.. code-block:: console + + $ igel --help + + +--------------------------------------------------------------------------------------------------------- + +Configuration Step +#################### + +First step is to provide a yaml file (you can also use json if you want) + +You can do this manually by creating a .yaml file (called igel.yaml by convention but you can name if whatever you want) +and editing it yourself. +However, if you are lazy (and you probably are, like me :D), you can use the igel init command to get started fast, +which will create a basic config file for you on the fly. + + + + +.. code-block:: console + + """ + igel init --help + + + Example: + If I want to use neural networks to classify whether someone is sick or not using the indian-diabetes dataset, + then I would use this command to initialize a yaml file n.b. you may need to rename outcome column in .csv to sick: + + $ igel init -type "classification" -model "NeuralNetwork" -target "sick" + """ + $ igel init + +After running the command, an igel.yaml file will be created for you in the current working directory. You can +check it out and modify it if you want to, otherwise you can also create everything from scratch. + +- Demo: + +.. image:: ../assets/igel-init.gif + +----------------------------------------------------------------------------------------------------------- + +.. code-block:: yaml + + # model definition + model: + # in the type field, you can write the type of problem you want to solve. Whether regression, classification or clustering + # Then, provide the algorithm you want to use on the data. Here I'm using the random forest algorithm + type: classification + algorithm: RandomForest # make sure you write the name of the algorithm in pascal case + arguments: + n_estimators: 100 # here, I set the number of estimators (or trees) to 100 + max_depth: 30 # set the max_depth of the tree + + # target you want to predict + # Here, as an example, I'm using the famous indians-diabetes dataset, where I want to predict whether someone have diabetes or not. + # Depending on your data, you need to provide the target(s) you want to predict here + target: + - sick + +In the example above, I'm using random forest to classify whether someone have +diabetes or not depending on some features in the dataset +I used the famous indian diabetes in this example `indian-diabetes dataset `_) + +Notice that I passed :code:`n_estimators` and :code:`max_depth` as additional arguments to the model. +If you don't provide arguments then the default will be used. +You don't have to memorize the arguments for each model. You can always run :code:`igel models` in your terminal, which will +get you to interactive mode, where you will be prompted to enter the model you want to use and type of the problem +you want to solve. Igel will then show you information about the model and a link that you can follow to see +a list of available arguments and how to use these. + +Training +######### + +- The expected way to use igel is from terminal (igel CLI): + +Run this command in terminal to fit/train a model, where you provide the **path to your dataset** and the **path to the yaml file** + +.. code-block:: console + + $ igel fit --data_path 'path_to_your_csv_dataset.csv' --yaml_path 'path_to_your_yaml_file.yaml' + + # or shorter + + $ igel fit -dp 'path_to_your_csv_dataset.csv' -yml 'path_to_your_yaml_file.yaml' + + """ + That's it. Your "trained" model can be now found in the model_results folder + (automatically created for you in your current working directory). + Furthermore, a description can be found in the description.json file inside the model_results folder. + """ + +- Demo: + +.. image:: ../assets/igel-fit.gif + +-------------------------------------------------------------------------------------------------------- + +Evaluation +################### + +You can then evaluate the trained/pre-fitted model: + +.. code-block:: console + + $ igel evaluate -dp 'path_to_your_evaluation_dataset.csv' + """ + This will automatically generate an evaluation.json file in the current directory, where all evaluation results are stored + """ + +- Demo: + +.. image:: ../assets/igel-eval.gif + +------------------------------------------------------------------------------------------------------ + +Prediction +######################### + +Finally, you can use the trained/pre-fitted model to make predictions if you are happy with the evaluation results: + +.. code-block:: console + + $ igel predict -dp 'path_to_your_test_dataset.csv' + """ + This will generate a predictions.csv file in your current directory, where all predictions are stored in a csv file + """ + +- Demo: + +.. image:: ../assets/igel-pred.gif + +.. image:: ../assets/igel-predict.gif + +---------------------------------------------------------------------------------------------------------- + +Experiment +#################### + +You can combine the train, evaluate and predict phases using one single command called experiment: + +.. code-block:: console + + $ igel experiment -DP "path_to_train_data path_to_eval_data path_to_test_data" -yml "path_to_yaml_file" + + """ + This will run fit using train_data, evaluate using eval_data and further generate predictions using the test_data + """ + +- Demo: + +.. image:: ../assets/igel-experiment.gif + +---------------------------------------------------------------------------------------------------------- + +Export +#################### + +You can export the trained/pre-fitted sklearn model into ONNX: + +.. code-block:: console + + $ igel export -dp "path_to_pre-fitted_sklearn_model" + + """ + This will convert the sklearn model into ONNX + """ + + +Use igel from python (instead of terminal) +########################################### + +- Alternatively, you can also write code if you want to: + +.. code-block:: python + + from igel import Igel + + Igel(cmd="fit", data_path="path_to_your_dataset", yaml_path="path_to_your_yaml_file") + """ + check the examples folder for more + """ + + +---------------------------------------------------------------------------------------------------------- + +Serve the model +################# + +The next step is to use your model in production. Igel helps you with this task too by providing the serve command. +Running the serve command will tell igel to serve your model. Precisely, igel will automatically build +a REST server and serve your model on a specific host and port, which you can configure by passing these as +cli options. + +The easiest way is to run: + +.. code-block:: console + + $ igel serve --model_results_dir "path_to_model_results_directory" + +Notice that igel needs the **--model_results_dir** or shortly -res_dir cli option in order to load the model and start the server. +By default, igel will serve your model on **localhost:8000**, however, you can easily override this by providing a host +and a port cli options. + +.. code-block:: console + + $ igel serve --model_results_dir "path_to_model_results_directory" --host "127.0.0.1" --port 8000 + +Igel uses `FastAPI `_ for creating the REST server, which is a modern high performance +framework +and `uvicorn `_ to run it under the hood. + +---------------------------------------------------------------------------------------------------------- + +Using the API with the served model +################################### + +This example was done using a pre-trained model (created by running igel init --target sick -type classification) and the Indian Diabetes dataset under examples/data. The headers of the columns in the original CSV are 'preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi' and 'age'. + +**CURL:** + + +- Post with single entry for each predictor + +.. code-block:: console + + $ curl -X POST localhost:8080/predict --header "Content-Type:application/json" -d '{"preg": 1, "plas": 180, "pres": 50, "skin": 12, "test": 1, "mass": 456, "pedi": 0.442, "age": 50}' + + Outputs: {"prediction":[[0.0]]} + +- Post with multiple options for each predictor + +.. code-block:: console + + $ curl -X POST localhost:8080/predict --header "Content-Type:application/json" -d '{"preg": [1, 6, 10], "plas":[192, 52, 180], "pres": [40, 30, 50], "skin": [25, 35, 12], "test": [0, 1, 1], "mass": [456, 123, 155], "pedi": [0.442, 0.22, 0.19], "age": [50, 40, 29]}' + + Outputs: {"prediction":[[1.0],[0.0],[0.0]]} + +**Caveats/Limitations:** + +- each predictor used to train the model must make an appearance in your data (i.e. don't leave any columns out) +- each list must have the same number of elements or you'll get an Internal Server Error +- as an extension of this, you cannot mix single elements and lists (i.e. {"plas": 0, "pres": [1, 2]} isn't allowed) +- the predict function takes a data path arg and reads in the data for you but with serving and calling your served model, you'll have to parse the data into JSON yourself however, the python client provided in `examples/python_client.py` will do that for you + +**Example usage of the Python Client:** + +.. code-block:: python + + from python_client import IgelClient + + # the client allows additional args with defaults: + # scheme="http", endpoint="predict", missing_values="mean" + client = IgelClient(host='localhost', port=8080) + + # you can post other types of files compatible with what Igel data reading allows + client.post("my_batch_file_for_predicting.csv") + + Outputs: : {"prediction":[[1.0],[0.0],[0.0]]} + +---------------------------------------------------------------------------------------------------------- + + +Overview +---------- +The main goal of igel is to provide you with a way to train/fit, evaluate and use models without writing code. +Instead, all you need is to provide/describe what you want to do in a simple yaml file. + +Basically, you provide description or rather configurations in the yaml file as key value pairs. +Here is an overview of all supported configurations (for now): + +.. code-block:: yaml + + # dataset operations + dataset: + type: csv # [str] -> type of your dataset + read_data_options: # options you want to supply for reading your data (See the detailed overview about this in the next section) + sep: # [str] -> Delimiter to use. + delimiter: # [str] -> Alias for sep. + header: # [int, list of int] -> Row number(s) to use as the column names, and the start of the data. + names: # [list] -> List of column names to use + index_col: # [int, str, list of int, list of str, False] -> Column(s) to use as the row labels of the DataFrame, + usecols: # [list, callable] -> Return a subset of the columns + squeeze: # [bool] -> If the parsed data only contains one column then return a Series. + prefix: # [str] -> Prefix to add to column numbers when no header, e.g. 'X' for X0, X1, ... + mangle_dupe_cols: # [bool] -> Duplicate columns will be specified as 'X', 'X.1', ...'X.N', rather than 'X'...'X'. Passing in False will cause data to be overwritten if there are duplicate names in the columns. + dtype: # [Type name, dict maping column name to type] -> Data type for data or columns + engine: # [str] -> Parser engine to use. The C engine is faster while the python engine is currently more feature-complete. + converters: # [dict] -> Dict of functions for converting values in certain columns. Keys can either be integers or column labels. + true_values: # [list] -> Values to consider as True. + false_values: # [list] -> Values to consider as False. + skipinitialspace: # [bool] -> Skip spaces after delimiter. + skiprows: # [list-like] -> Line numbers to skip (0-indexed) or number of lines to skip (int) at the start of the file. + skipfooter: # [int] -> Number of lines at bottom of file to skip + nrows: # [int] -> Number of rows of file to read. Useful for reading pieces of large files. + na_values: # [scalar, str, list, dict] -> Additional strings to recognize as NA/NaN. + keep_default_na: # [bool] -> Whether or not to include the default NaN values when parsing the data. + na_filter: # [bool] -> Detect missing value markers (empty strings and the value of na_values). In data without any NAs, passing na_filter=False can improve the performance of reading a large file. + verbose: # [bool] -> Indicate number of NA values placed in non-numeric columns. + skip_blank_lines: # [bool] -> If True, skip over blank lines rather than interpreting as NaN values. + parse_dates: # [bool, list of int, list of str, list of lists, dict] -> try parsing the dates + infer_datetime_format: # [bool] -> If True and parse_dates is enabled, pandas will attempt to infer the format of the datetime strings in the columns, and if it can be inferred, switch to a faster method of parsing them. + keep_date_col: # [bool] -> If True and parse_dates specifies combining multiple columns then keep the original columns. + dayfirst: # [bool] -> DD/MM format dates, international and European format. + cache_dates: # [bool] -> If True, use a cache of unique, converted dates to apply the datetime conversion. + thousands: # [str] -> the thousands operator + decimal: # [str] -> Character to recognize as decimal point (e.g. use ',' for European data). + lineterminator: # [str] -> Character to break file into lines. + escapechar: # [str] -> One-character string used to escape other characters. + comment: # [str] -> Indicates remainder of line should not be parsed. If found at the beginning of a line, the line will be ignored altogether. This parameter must be a single character. + encoding: # [str] -> Encoding to use for UTF when reading/writing (ex. 'utf-8'). + dialect: # [str, csv.Dialect] -> If provided, this parameter will override values (default or not) for the following parameters: delimiter, doublequote, escapechar, skipinitialspace, quotechar, and quoting + delim_whitespace: # [bool] -> Specifies whether or not whitespace (e.g. ' ' or ' ') will be used as the sep + low_memory: # [bool] -> Internally process the file in chunks, resulting in lower memory use while parsing, but possibly mixed type inference. + memory_map: # [bool] -> If a filepath is provided for filepath_or_buffer, map the file object directly onto memory and access the data directly from there. Using this option can improve performance because there is no longer any I/O overhead. + + random_numbers: # random numbers options in case you wanted to generate the same random numbers on each run + generate_reproducible: # [bool] -> set this to true to generate reproducible results + seed: # [int] -> the seed number is optional. A seed will be set up for you if you didn't provide any + + split: # split options + test_size: 0.2 #[float] -> 0.2 means 20% for the test data, so 80% are automatically for training + shuffle: true # [bool] -> whether to shuffle the data before/while splitting + stratify: None # [list, None] -> If not None, data is split in a stratified fashion, using this as the class labels. + + preprocess: # preprocessing options + missing_values: mean # [str] -> other possible values: [drop, median, most_frequent, constant] check the docs for more + encoding: + type: oneHotEncoding # [str] -> other possible values: [labelEncoding] + scale: # scaling options + method: standard # [str] -> standardization will scale values to have a 0 mean and 1 standard deviation | you can also try minmax + target: inputs # [str] -> scale inputs. | other possible values: [outputs, all] # if you choose all then all values in the dataset will be scaled + + + # model definition + model: + type: classification # [str] -> type of the problem you want to solve. | possible values: [regression, classification, clustering] + algorithm: NeuralNetwork # [str (notice the pascal case)] -> which algorithm you want to use. | type igel algorithms in the Terminal to know more + arguments: # model arguments: you can check the available arguments for each model by running igel help in your terminal + use_cv_estimator: false # [bool] -> if this is true, the CV class of the specific model will be used if it is supported + cross_validate: + cv: # [int] -> number of kfold (default 5) + n_jobs: # [signed int] -> The number of CPUs to use to do the computation (default None) + verbose: # [int] -> The verbosity level. (default 0) + hyperparameter_search: + method: grid_search # method you want to use: grid_search and random_search are supported + parameter_grid: # put your parameters grid here that you want to use, an example is provided below + param1: [val1, val2] + param2: [val1, val2] + arguments: # additional arguments you want to provide for the hyperparameter search + cv: 5 # number of folds + refit: true # whether to refit the model after the search + return_train_score: false # whether to return the train score + verbose: 0 # verbosity level + + # target you want to predict + target: # list of strings: basically put here the column(s), you want to predict that exist in your csv dataset + - put the target you want to predict here + - you can assign many target if you are making a multioutput prediction + +Read Data Options +------------------ + +.. note:: + igel uses pandas under the hood to read & parse the data. Hence, you can + find this data optional parameters also in the pandas official documentation. + +A detailed overview of the configurations you can provide in the yaml (or json) file is given below. +Notice that you will certainly not need all the configuration values for the dataset. They are optional. +Generally, igel will figure out how to read your dataset. + +However, you can help it by providing extra fields using this read_data_options section. +For example, one of the helpful values in my opinion is the "sep", which defines how your columns +in the csv dataset are separated. Generally, csv datasets are separated by commas, which is also the default value +here. However, it may be separated by a semicolon in your case. + +Hence, you can provide this in the read_data_options. Just add the :code:`sep: ";"` under read_data_options. + + + +.. list-table:: Supported Read Data Options + :widths: 25 25 50 + :header-rows: 1 + + * - Parameter + - Type + - Explanation + * - sep + - str, default ',' + - Delimiter to use. If sep is None, the C engine cannot automatically detect the separator, but the Python parsing engine can, meaning the latter will be used and automatically detect the separator by Python's builtin sniffer tool, csv.Sniffer. In addition, separators longer than 1 character and different from '\s+' will be interpreted as regular expressions and will also force the use of the Python parsing engine. Note that regex delimiters are prone to ignoring quoted data. Regex example: '\r\t'. + * - delimiter + - default None + - Alias for sep. + * - header + - int, list of int, default 'infer' + - Row number(s) to use as the column names, and the start of the data. Default behavior is to infer the column names: if no names are passed the behavior is identical to header=0 and column names are inferred from the first line of the file, if column names are passed explicitly then the behavior is identical to header=None. Explicitly pass header=0 to be able to replace existing names. The header can be a list of integers that specify row locations for a multi-index on the columns e.g. [0,1,3]. Intervening rows that are not specified will be skipped (e.g. 2 in this example is skipped). Note that this parameter ignores commented lines and empty lines if skip_blank_lines=True, so header=0 denotes the first line of data rather than the first line of the file. + * - names + - array-like, optional + - List of column names to use. If the file contains a header row, then you should explicitly pass header=0 to override the column names. Duplicates in this list are not allowed. + * - index_col + - int, str, sequence of int / str, or False, default None + - Column(s) to use as the row labels of the DataFrame, either given as string name or column index. If a sequence of int / str is given, a MultiIndex is used. Note: index_col=False can be used to force pandas to not use the first column as the index, e.g. when you have a malformed file with delimiters at the end of each line. + * - usecols + - list-like or callable, optional + - Return a subset of the columns. If list-like, all elements must either be positional (i.e. integer indices into the document columns) or strings that correspond to column names provided either by the user in names or inferred from the document header row(s). For example, a valid list-like usecols parameter would be [0, 1, 2] or ['foo', 'bar', 'baz']. Element order is ignored, so usecols=[0, 1] is the same as [1, 0]. To instantiate a DataFrame from data with element order preserved use pd.read_csv(data, usecols=['foo', 'bar'])[['foo', 'bar']] for columns in ['foo', 'bar'] order or pd.read_csv(data, usecols=['foo', 'bar'])[['bar', 'foo']] for ['bar', 'foo'] order. If callable, the callable function will be evaluated against the column names, returning names where the callable function evaluates to True. An example of a valid callable argument would be lambda x: x.upper() in ['AAA', 'BBB', 'DDD']. Using this parameter results in much faster parsing time and lower memory usage. + * - squeeze + - bool, default False + - If the parsed data only contains one column then return a Series. + + * - prefix + - str, optional + - Prefix to add to column numbers when no header, e.g. 'X' for X0, X1, ... + * - mangle_dupe_cols + - bool, default True + - Duplicate columns will be specified as 'X', 'X.1', ...'X.N', rather than 'X'...'X'. Passing in False will cause data to be overwritten if there are duplicate names in the columns. + * - dtype + - {'c', 'python'}, optional + - Parser engine to use. The C engine is faster while the python engine is currently more feature-complete. + * - converters + - dict, optional + - Dict of functions for converting values in certain columns. Keys can either be integers or column labels. + * - true_values + - list, optional + - Values to consider as True. + + * - false_values + - list, optional + - Values to consider as False. + * - skipinitialspace + - bool, default False + - Skip spaces after delimiter. + * - skiprows + - list-like, int or callable, optional + - Line numbers to skip (0-indexed) or number of lines to skip (int) at the start of the file. If callable, the callable function will be evaluated against the row indices, returning True if the row should be skipped and False otherwise. An example of a valid callable argument would be lambda x: x in [0, 2]. + * - skipfooter + - int, default 0 + - Number of lines at bottom of file to skip (Unsupported with engine='c'). + * - nrows + - int, optional + - Number of rows of file to read. Useful for reading pieces of large files. + * - na_values + - scalar, str, list-like, or dict, optional + - Additional strings to recognize as NA/NaN. If dict passed, specific per-column NA values. By default the following values are interpreted as NaN: '', '#N/A', '#N/A N/A', '#NA', '-1.#IND', '-1.#QNAN', '-NaN', '-nan', '1.#IND', '1.#QNAN', '', 'N/A', 'NA', 'NULL', 'NaN', 'n/a', 'nan', 'null'. + * - keep_default_na + - bool, default True + - Whether or not to include the default NaN values when parsing the data. Depending on whether na_values is passed in, the behavior is as follows: If keep_default_na is True, and na_values are specified, na_values is appended to the default NaN values used for parsing. If keep_default_na is True, and na_values are not specified, only the default NaN values are used for parsing. If keep_default_na is False, and na_values are specified, only the NaN values specified na_values are used for parsing. If keep_default_na is False, and na_values are not specified, no strings will be parsed as NaN. Note that if na_filter is passed in as False, the keep_default_na and na_values parameters will be ignored. + * - na_filter + - bool, default True + - Detect missing value markers (empty strings and the value of na_values). In data without any NAs, passing na_filter=False can improve the performance of reading a large file. + * - verbose + - bool, default False + - Indicate number of NA values placed in non-numeric columns. + * - skip_blank_lines + - bool, default True + - If True, skip over blank lines rather than interpreting as NaN values. + * - parse_dates + - bool or list of int or names or list of lists or dict, default False + - The behavior is as follows: boolean. If True -> try parsing the index. list of int or names. e.g. If [1, 2, 3] -> try parsing columns 1, 2, 3 each as a separate date column. list of lists. e.g. If [[1, 3]] -> combine columns 1 and 3 and parse as a single date column. dict, e.g. {'foo' : [1, 3]} -> parse columns 1, 3 as date and call result 'foo' If a column or index cannot be represented as an array of datetimes, say because of an unparseable value or a mixture of timezones, the column or index will be returned unaltered as an object data type. + * - infer_datetime_format + - bool, default False + - If True and parse_dates is enabled, pandas will attempt to infer the format of the datetime strings in the columns, and if it can be inferred, switch to a faster method of parsing them. In some cases this can increase the parsing speed by 5-10x. + * - keep_date_col + - bool, default False + - If True and parse_dates specifies combining multiple columns then keep the original columns. + * - date_parser + - function, optional + - Function to use for converting a sequence of string columns to an array of datetime instances. The default uses dateutil.parser.parser to do the conversion. Pandas will try to call date_parser in three different ways, advancing to the next if an exception occurs: 1) Pass one or more arrays (as defined by parse_dates) as arguments; 2) concatenate (row-wise) the string values from the columns defined by parse_dates into a single array and pass that; and 3) call date_parser once for each row using one or more strings (corresponding to the columns defined by parse_dates) as arguments. + * - dayfirst + - bool, default False + - DD/MM format dates, international and European format. + + * - cache_dates + - bool, default True + - If True, use a cache of unique, converted dates to apply the datetime conversion. May produce significant speed-up when parsing duplicate date strings, especially ones with timezone offsets. + * - thousands + - str, optional + - Thousands separator. + * - decimal + - str, default '.' + - Character to recognize as decimal point (e.g. use ',' for European data). + * - lineterminator + - str (length 1), optional + - Character to break file into lines. Only valid with C parser. + * - escapechar + - str (length 1), optional + - One-character string used to escape other characters. + * - comment + - str, optional + - Indicates remainder of line should not be parsed. If found at the beginning of a line, the line will be ignored altogether. + * - encoding + - str, optional + - Encoding to use for UTF when reading/writing (ex. 'utf-8'). + * - dialect + - str or csv.Dialect, optional + - If provided, this parameter will override values (default or not) for the following parameters: delimiter, doublequote, escapechar, skipinitialspace, quotechar, and quoting + * - low_memory + - bool, default True + - Internally process the file in chunks, resulting in lower memory use while parsing, but possibly mixed type inference. To ensure no mixed types either set False, or specify the type with the dtype parameter. Note that the entire file is read into a single DataFrame regardless, + * - memory_map + - bool, default False + - map the file object directly onto memory and access the data directly from there. Using this option can improve performance because there is no longer any I/O overhead. + + +E2E Example +----------- + +A complete end to end solution is provided in this section to prove the capabilities of **igel**. +As explained previously, you need to create a yaml configuration file. Here is an end to end example for +predicting whether someone have diabetes or not using the **decision tree** algorithm. The dataset can be found in the examples folder. + +- **Fit/Train a model**: + +.. code-block:: yaml + + model: + type: classification + algorithm: DecisionTree + + target: + - sick + +.. code-block:: console + + $ igel fit -dp path_to_the_dataset -yml path_to_the_yaml_file + +That's it, igel will now fit the model for you and save it in a model_results folder in your current directory. + + +- **Evaluate the model**: + +Evaluate the pre-fitted model. Igel will load the pre-fitted model from the model_results directory and evaluate it for you. +You just need to run the evaluate command and provide the path to your evaluation data. + +.. code-block:: console + + $ igel evaluate -dp path_to_the_evaluation_dataset + +That's it! Igel will evaluate the model and store statistics/results in an **evaluation.json** file inside the model_results folder + +- **Predict**: + +Use the pre-fitted model to predict on new data. This is done automatically by igel, you just need to provide the +path to your data that you want to use prediction on. + +.. code-block:: console + + $ igel predict -dp path_to_the_new_dataset + +That's it! Igel will use the pre-fitted model to make predictions and save it in a **predictions.csv** file inside the model_results folder + +Advanced Usage +--------------- + +You can also carry out some preprocessing methods or other operations by providing them in the yaml file. +Here is an example, where the data is split to 80% for training and 20% for validation/testing. +Also, the data are shuffled while splitting. + +Furthermore, the data are preprocessed by replacing missing values with the mean ( you can also use median, mode etc..). +check `this link `_ for more information + + +.. code-block:: yaml + + # dataset operations + dataset: + split: + test_size: 0.2 + shuffle: True + stratify: default + + preprocess: # preprocessing options + missing_values: mean # other possible values: [drop, median, most_frequent, constant] check the docs for more + encoding: + type: oneHotEncoding # other possible values: [labelEncoding] + scale: # scaling options + method: standard # standardization will scale values to have a 0 mean and 1 standard deviation | you can also try minmax + target: inputs # scale inputs. | other possible values: [outputs, all] # if you choose all then all values in the dataset will be scaled + + # model definition + model: + type: classification + algorithm: RandomForest + arguments: + # notice that this is the available args for the random forest model. check different available args for all supported models by running igel help + n_estimators: 100 + max_depth: 20 + + # target you want to predict + target: + - sick + +Then, you can fit the model by running the igel command as shown in the other examples + +.. code-block:: console + + $ igel fit -dp path_to_the_dataset -yml path_to_the_yaml_file + +For evaluation + +.. code-block:: console + + $ igel evaluate -dp path_to_the_evaluation_dataset + +For production + +.. code-block:: console + + $ igel predict -dp path_to_the_new_dataset + +Examples +---------- + +In the examples folder in the repository, you will find a data folder,where the famous indian-diabetes, iris dataset +and the linnerud (from sklearn) datasets are stored. +Furthermore, there are end to end examples inside each folder, where there are scripts and yaml files that +will help you get started. + + +The indian-diabetes-example folder contains two examples to help you get started: + +- The first example is using a **neural network**, where the configurations are stored in the neural-network.yaml file +- The second example is using a **random forest**, where the configurations are stored in the random-forest.yaml file + +The iris-example folder contains a **logistic regression** example, where some preprocessing (one hot encoding) +is conducted on the target column to show you more the capabilities of igel. + +Furthermore, the multioutput-example contains a **multioutput regression** example. +Finally, the cv-example contains an example using the Ridge classifier using cross validation. + +You can also find a cross validation and a hyperparameter search examples in the folder. + +I suggest you play around with the examples and igel cli. However, +you can also directly execute the fit.py, evaluate.py and predict.py if you want to. + +Auto ML Examples +------------------ + +ImageClassification +#################### + +First, create or modify a dataset of images that are categorized into sub-folders based on the image label/class +For example, if you are have dogs and cats images, then you will need 2 sub-folders: + +- folder 0, which contains cats images (here the label 0 indicates a cat) +- folder 1, which contains dogs images (here the label 1 indicates a dog) + +Assuming these two sub-folder are contained in one parent folder called images, just feed data to igel: + +.. code-block:: console + + $ igel auto-train -dp ./images --task ImageClassification + +Igel will handle everything from pre-processing the data to optimizing hyperparameters. At the end, +the best model will be stored in the current working dir. + + + +TextClassification +#################### + +First, create or modify a text dataset that are categorized into sub-folders based on the text label/class +For example, if you are have a text dataset of positive and negative feedbacks, then you will need 2 sub-folders: + +- folder 0, which contains negative feedbacks (here the label 0 indicates a negative one) +- folder 1, which contains positive feedbacks (here the label 1 indicates a positive one) + +Assuming these two sub-folder are contained in one parent folder called texts, just feed data to igel: + +.. code-block:: console + + $ igel auto-train -dp ./texts --task TextClassification + +Igel will handle everything from pre-processing the data to optimizing hyperparameters. At the end, +the best model will be stored in the current working dir. + + +GUI +---- + +You can also run the igel UI if you are not familiar with the terminal. Just install igel on your machine +as mentioned above. Then run this single command in your terminal + +.. code-block:: console + + $ igel gui + +This will open up the gui, which is very simple to use. Check examples of how the gui looks like and how to use it +here: https://github.com/nidhaloff/igel-ui + + +Running with Docker +-------------------- + +- Use the official image (recommended): + +You can pull the image first from docker hub + +.. code-block:: console + + $ docker pull nidhaloff/igel + +Then use it: + +.. code-block:: console + + $ docker run -it --rm -v $(pwd):/data nidhaloff/igel fit -yml 'your_file.yaml' -dp 'your_dataset.csv' + + +- Alternatively, you can create your own image locally if you want: + +You can run igel inside of docker by first building the image: + +.. code-block:: console + + $ docker build -t igel . + +And then running it and attaching your current directory (does not need to be the igel directory) as /data (the workdir) inside of the container: + +.. code-block:: console + + $ docker run -it --rm -v $(pwd):/data igel fit -yml 'your_file.yaml' -dp 'your_dataset.csv' + +Links +------ + +- Article: https://medium.com/@nidhalbacc/machine-learning-without-writing-code-984b238dd890 + + + +Help/GetHelp +--------------- + +If you are facing any problems, please feel free to open an issue. +Additionally, you can make contact with the author for further information/questions. + +Do you like igel? +You can always help the development of this project by: + +- Following on github and/or twitter +- Star the github repo +- Watch the github repo for new releases +- Tweet about the package +- Help others with issues on github +- Create issues and pull requests +- Sponsor the project + +Contributions +-------------- + +You think this project is useful and you want to bring new ideas, new features, bug fixes, extend the docs? + +Contributions are always welcome. +Make sure you read `the guidelines `_ first + +License +-------- + +MIT license + +Copyright (c) 2020-present, Nidhal Baccouri + +# Frequently Asked Questions (FAQ) + +## What is igel? + +igel is a machine learning tool that allows you to train, test, and use models without writing code. + +## How do I install igel? + +```bash +pip install igel +``` + +## What file formats does igel support for configuration? + +igel supports both YAML and JSON configuration files. + +## Where can I find example config files? + +See the `examples/` directory for sample YAML and JSON configs for different ML tasks. + +## How do I contribute to igel? + +Check out the [CONTRIBUTING.md](CONTRIBUTING.md) file for guidelines and tips for new contributors. + +## Where do I report bugs or request features? + +Please open an issue on the [GitHub Issues page](https://github.com/nidhaloff/igel/issues). + +## How do I get help? + +You can ask questions by opening an issue or joining the project discussions on GitHub. diff --git a/igel/docs/authors.rst b/igel/docs/authors.rst new file mode 100644 index 0000000..e122f91 --- /dev/null +++ b/igel/docs/authors.rst @@ -0,0 +1 @@ +.. include:: ../AUTHORS.rst diff --git a/igel/docs/conf.py b/igel/docs/conf.py new file mode 100644 index 0000000..d63c0a0 --- /dev/null +++ b/igel/docs/conf.py @@ -0,0 +1,180 @@ +#!/usr/bin/env python +# +# igel documentation build configuration file, created by +# sphinx-quickstart on Fri Jun 9 13:47:02 2017. +# +# This file is execfile()d with the current directory set to its +# containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +# If extensions (or modules to document with autodoc) are in another +# directory, add these directories to sys.path here. If the directory is +# relative to the documentation root, use os.path.abspath to make it +# absolute, like shown here. +# + +import os +import sys +sys.path.insert(0, os.path.abspath('..')) + +import igel + +# -- General configuration --------------------------------------------- + + +html_logo = '../assets/logo.jpg' + +# If your documentation needs a minimal Sphinx version, state it here. +# +# needs_sphinx = '1.0' +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. +extensions = ['sphinx_copybutton', 'sphinx.ext.autodoc', + 'sphinx.ext.viewcode', 'sphinx.ext.autosectionlabel'] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix(es) of source filenames. +# You can specify multiple suffix as a list of string: +# +# source_suffix = ['.rst', '.md'] +source_suffix = '.rst' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +project = 'igel' +copyright = "2020, Nidhal Baccouri" +author = "Nidhal Baccouri" + +# The version info for the project you're documenting, acts as replacement +# for |version| and |release|, also used in various other places throughout +# the built documents. +# +# The short X.Y version. +version = igel.__version__ +# The full version, including alpha/beta/rc tags. +release = igel.__version__ + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +# +# This is also used if you do content translation via gettext catalogs. +# Usually you set "language" from the command line for these cases. +language = None + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This patterns also effect to html_static_path and html_extra_path +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# If true, `todo` and `todoList` produce output, else they produce nothing. +todo_include_todos = False + + +# -- Options for HTML output ------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'default' + +# Theme options are theme-specific and customize the look and feel of a +# theme further. For a list of options available for each theme, see the +# documentation. +# +html_theme_options = { + 'canonical_url': '', + 'analytics_id': 'UA-XXXXXXX-1', # Provided by Google in your dashboard + 'logo_only': False, + 'display_version': True, + 'prev_next_buttons_location': 'bottom', + 'style_external_links': False, + 'vcs_pageview_mode': '', + 'style_nav_header_background': 'white', + # Toc options + 'collapse_navigation': True, + 'sticky_navigation': True, + 'navigation_depth': 4, + 'includehidden': True, + 'titles_only': False +} + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + + +# -- Options for HTMLHelp output --------------------------------------- + +# Output file base name for HTML help builder. +htmlhelp_basename = 'igeldoc' + + +# -- Options for LaTeX output ------------------------------------------ + +latex_elements = { + # The paper size ('letterpaper' or 'a4paper'). + # + # 'papersize': 'letterpaper', + + # The font size ('10pt', '11pt' or '12pt'). + # + # 'pointsize': '10pt', + + # Additional stuff for the LaTeX preamble. + # + # 'preamble': '', + + # Latex figure (float) alignment + # + # 'figure_align': 'htbp', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, author, documentclass +# [howto, manual, or own class]). +latex_documents = [ + (master_doc, 'igel.tex', + 'igel Documentation', + 'Nidhal Baccouri', 'manual'), +] + + +# -- Options for manual page output ------------------------------------ + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + (master_doc, 'igel', + 'igel Documentation', + [author], 1) +] + + +# -- Options for Texinfo output ---------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + (master_doc, 'igel', + 'igel Documentation', + author, + 'igel', + 'One line description of project.', + 'Miscellaneous'), +] + +# Minor update: contribution trigger for PR diff --git a/igel/docs/contributing.rst b/igel/docs/contributing.rst new file mode 100644 index 0000000..e582053 --- /dev/null +++ b/igel/docs/contributing.rst @@ -0,0 +1 @@ +.. include:: ../CONTRIBUTING.rst diff --git a/igel/docs/history.rst b/igel/docs/history.rst new file mode 100644 index 0000000..2506499 --- /dev/null +++ b/igel/docs/history.rst @@ -0,0 +1 @@ +.. include:: ../HISTORY.rst diff --git a/igel/docs/index.rst b/igel/docs/index.rst new file mode 100644 index 0000000..bc793de --- /dev/null +++ b/igel/docs/index.rst @@ -0,0 +1,20 @@ +Welcome to igel's documentation! +====================================== + +.. toctree:: + :maxdepth: 2 + :caption: Contents: + + README + installation + usage + modules + contributing + authors + history + +Indices and tables +================== +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` diff --git a/igel/docs/installation.rst b/igel/docs/installation.rst new file mode 100644 index 0000000..fa84cca --- /dev/null +++ b/igel/docs/installation.rst @@ -0,0 +1,51 @@ +.. highlight:: shell + +============ +Installation +============ + + +Stable release +-------------- + +To install igel, run this command in your terminal: + +.. code-block:: console + + $ pip install igel + +This is the preferred method to install igel, as it will always install the most recent stable release. + +If you don't have `pip`_ installed, this `Python installation guide`_ can guide +you through the process. + +.. _pip: https://pip.pypa.io +.. _Python installation guide: http://docs.python-guide.org/en/latest/starting/installation/ + + +From sources +------------ + +The sources for igel can be downloaded from the `Github repo`_. + +You can either clone the public repository: + +.. code-block:: console + + $ git clone git://github.com/nidhaloff/igel + +Or download the `tarball`_: + +.. code-block:: console + + $ curl -OJL https://github.com/nidhaloff/igel/tarball/master + +Once you have a copy of the source, you can install it with: + +.. code-block:: console + + $ python setup.py install + + +.. _Github repo: https://github.com/nidhaloff/igel +.. _tarball: https://github.com/nidhaloff/igel/tarball/master diff --git a/igel/docs/make.bat b/igel/docs/make.bat new file mode 100644 index 0000000..7fc955a --- /dev/null +++ b/igel/docs/make.bat @@ -0,0 +1,36 @@ +@ECHO OFF + +pushd %~dp0 + +REM Command file for Sphinx documentation + +if "%SPHINXBUILD%" == "" ( + set SPHINXBUILD=python -msphinx +) +set SOURCEDIR=. +set BUILDDIR=_build +set SPHINXPROJ=igel + +if "%1" == "" goto help + +%SPHINXBUILD% >NUL 2>NUL +if errorlevel 9009 ( + echo. + echo.The Sphinx module was not found. Make sure you have Sphinx installed, + echo.then set the SPHINXBUILD environment variable to point to the full + echo.path of the 'sphinx-build' executable. Alternatively you may add the + echo.Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.http://sphinx-doc.org/ + exit /b 1 +) + +%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% +goto end + +:help +%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% + +:end +popd diff --git a/igel/docs/requirements_docs.txt b/igel/docs/requirements_docs.txt new file mode 100644 index 0000000..4f03bd6 --- /dev/null +++ b/igel/docs/requirements_docs.txt @@ -0,0 +1,19 @@ +pip==20.2.2 +bump2version==1.0.0 +wheel==0.35.1 +watchdog==0.10.3 +flake8==3.8.3 +tox==3.19.0 +coverage==5.2.1 +Sphinx==3.2.1 +twine==3.2.0 +Click==7.1.2 +pytest==6.0.1 +pytest-runner==5.2 +pandas==1.1.1 +PyYAML==5.3.1 +scikit-learn==0.23.2 +sphinx-copybutton==0.3.1 +sphinx_glpi_theme==0.3 +importlib-metadata==1.6.0 +skl2onnx==1.10.3 diff --git a/igel/docs/usage.rst b/igel/docs/usage.rst new file mode 100644 index 0000000..cdfe722 --- /dev/null +++ b/igel/docs/usage.rst @@ -0,0 +1,360 @@ +===== +Usage +===== + + +you can run this command to get instruction on how to use the model: + +.. code-block:: console + + $ igel --help + + # or just + + $ igel -h + """ + Take some time and read the output of help command. You ll save time later if you understand how to use igel. + """ + + +First step is to provide a yaml file. You can do this manually by creating a .yaml file and editing it yourself. +However, if you are lazy (and you probably are, like me :D), you can use the igel init command to get started fast: + +.. code-block:: console + + """ + igel init + possible optional args are: (notice that these args are optional, so you can also just run igel init if you want) + -type: regression or classification + -model: model you want to use + -target: target you want to predict + + + Example: + If I want to use neural networks to classify whether someone is sick or not using the indian-diabetes dataset, + then I would use this command to initliaze a yaml file: + $ igel init -type "classification" -model "NeuralNetwork" -target "sick" + """ + $ igel init + +After running the command, an igel.yaml file will be created for you in the current working directory. You can +check it out and modify it if you want to, otherwise you can also create everything from scratch. + + +.. code-block:: yaml + + # model definition + model: + # in the type field, you can write the type of problem you want to solve. Whether regression or classification + # Then, provide the algorithm you want to use on the data. Here I'm using the random forest algorithm + type: classification + algorithm: RandomForest # make sure you write the name of the algorithm in pascal case + arguments: + n_estimators: 100 # here, I set the number of estimators (or trees) to 100 + max_depth: 30 # set the max_depth of the tree + + # target you want to predict + # Here, as an example, I'm using the famous indians-diabetes dataset, where I want to predict whether someone have diabetes or not. + # Depending on your data, you need to provide the target(s) you want to predict here + target: + - sick + +In the example above, I'm using random forest to classify whether someone have +diabetes or not depending on some features in the dataset +I used this `indian-diabetes dataset `_) + + +- The expected way to use igel is from terminal (igel CLI): + +Run this command in terminal to fit/train a model, where you provide the **path to your dataset** and the **path to the yaml file** + +.. code-block:: console + + $ igel fit --data_path 'path_to_your_csv_dataset.csv' --yaml_file 'path_to_your_yaml_file.yaml' + + # or shorter + + $ igel fit -dp 'path_to_your_csv_dataset.csv' -yml 'path_to_your_yaml_file.yaml' + + """ + That's it. Your "trained" model can be now found in the model_results folder + (automatically created for you in your current working directory). + Furthermore, a description can be found in the description.json file inside the model_results folder. + """ + +You can then evaluate the trained/pre-fitted model: + +.. code-block:: console + + $ igel evaluate -dp 'path_to_your_evaluation_dataset.csv' + """ + This will automatically generate an evaluation.json file in the current directory, where all evaluation results are stored + """ + +Finally, you can use the trained/pre-fitted model to make predictions if you are happy with the evaluation results: + +.. code-block:: console + + $ igel predict -dp 'path_to_your_test_dataset.csv' + """ + This will generate a predictions.csv file in your current directory, where all predictions are stored in a csv file + """ + +You can combine the train, evaluate and predict phases using one single command called experiment: + +.. code-block:: console + + $ igel experiment -DP "path_to_train_data path_to_eval_data path_to_test_data" -yml "path_to_yaml_file" + + """ + This will run fit using train_data, evaluate using eval_data and further generate predictions using the test_data + """ + +You can export the trained/pre-fitted sklearn model into ONNX: + +.. code-block:: console + + $ igel export -dp "path_to_trained_model" + + """ + This will convert the sklearn model into ONNX + """ + +- Alternatively, you can also write code if you want to: + +.. code-block:: python + + from igel import Igel + + # provide the arguments in a dictionary + params = { + 'cmd': 'fit', # provide the command you want to use. whether fit, evaluate or predict + 'data_path': 'path_to_your_dataset', + 'yaml_path': 'path_to_your_yaml_file' + } + + Igel(**params) + """ + check the examples folder for more + """ + +Overview +---------- +The main goal of igel is to provide you with a way to train/fit, evaluate and use models without writing code. +Instead, all you need is to provide/describe what you want to do in a simple yaml file. + +Basically, you provide description or rather configurations in the yaml file as key value pairs. +Here is an overview of all supported configurations (for now): + +.. code-block:: yaml + + # dataset operations + dataset: + type: csv + read_data_options: default + split: # split options + test_size: 0.2 # 0.2 means 20% for the test data, so 80% are automatically for training + shuffle: True # whether to shuffle the data before/while splitting + stratify: None # If not None, data is split in a stratified fashion, using this as the class labels. + + preprocess: # preprocessing options + missing_values: mean # other possible values: [drop, median, most_frequent, constant] check the docs for more + encoding: + type: oneHotEncoding # other possible values: [labelEncoding] + scale: # scaling options + method: standard # standardization will scale values to have a 0 mean and 1 standard deviation | you can also try minmax + target: inputs # scale inputs. | other possible values: [outputs, all] # if you choose all then all values in the dataset will be scaled + + + # model definition + model: + type: classification # type of the problem you want to solve. | possible values: [regression, classification] + algorithm: NeuralNetwork # which algorithm you want to use. | type igel algorithms in the Terminal to know more + arguments: default # model arguments: you can check the available arguments for each model by running igel help in your terminal + + # target you want to predict + target: + - put the target you want to predict here + - you can assign many target if you are making a multioutput prediction + + +E2E Example +----------- + +A complete end to end solution is provided in this section to prove the capabilities of **igel**. +As explained previously, you need to create a yaml configuration file. Here is an end to end example for +predicting whether someone have diabetes or not using the **decision tree** algorithm. The dataset can be found in the examples folder. + +- **Fit/Train a model**: + +.. code-block:: yaml + + model: + type: classification + algorithm: DecisionTree + + target: + - sick + +.. code-block:: console + + $ igel fit -dp path_to_the_dataset -yml path_to_the_yaml_file + +That's it, igel will now fit the model for you and save it in a model_results folder in your current directory. + + +- **Evaluate the model**: + +Evaluate the pre-fitted model. Igel will load the pre-fitted model from the model_results directory and evaluate it for you. +You just need to run the evaluate command and provide the path to your evaluation data. + +.. code-block:: console + + $ igel evaluate -dp path_to_the_evaluation_dataset + +That's it! Igel will evaluate the model and store statistics/results in an **evaluation.json** file inside the model_results folder + +- **Predict**: + +Use the pre-fitted model to predict on new data. This is done automatically by igel, you just need to provide the +path to your data that you want to use prediction on. + +.. code-block:: console + + $ igel predict -dp path_to_the_new_dataset + +That's it! Igel will use the pre-fitted model to make predictions and save it in a **predictions.csv** file inside the model_results folder + +Advanced Usage +--------------- + +You can also carry out some preprocessing methods or other operations by providing the it in the yaml file. +Here is an example, where the data is split to 80% for training and 20% for validation/testing. +Also, the data are shuffled while splitting. + +Furthermore, the data are preprocessed by replacing missing values with the mean ( you can also use median, mode etc..). +check `this link `_ for more information + + +.. code-block:: yaml + + # dataset operations + dataset: + split: + test_size: 0.2 + shuffle: True + stratify: default + + preprocess: # preprocessing options + missing_values: mean # other possible values: [drop, median, most_frequent, constant] check the docs for more + encoding: + type: oneHotEncoding # other possible values: [labelEncoding] + scale: # scaling options + method: standard # standardization will scale values to have a 0 mean and 1 standard deviation | you can also try minmax + target: inputs # scale inputs. | other possible values: [outputs, all] # if you choose all then all values in the dataset will be scaled + + # model definition + model: + type: classification + algorithm: RandomForest + arguments: + # notice that this is the available args for the random forest model. check different available args for all supported models by running igel help + n_estimators: 100 + max_depth: 20 + + # target you want to predict + target: + - sick + +Then, you can fit the model by running the igel command as shown in the other examples + +.. code-block:: console + + $ igel fit -dp path_to_the_dataset -yml path_to_the_yaml_file + +For evaluation + +.. code-block:: console + + $ igel evaluate -dp path_to_the_evaluation_dataset + +For production + +.. code-block:: console + + $ igel predict -dp path_to_the_new_dataset + +Examples +---------- + +In the examples folder in the repository, you will find a data folder,where the famous indian-diabetes, iris dataset +and the linnerud (from sklearn) datasets are stored. +Furthermore, there are end to end examples inside each folder, where there are scripts and yaml files that +will help you get started. + + +The indian-diabetes-example folder contains two examples to help you get started: + +- The first example is using a **neural network**, where the configurations are stored in the neural-network.yaml file +- The second example is using a **random forest**, where the configurations are stored in the random-forest.yaml file + +The iris-example folder contains a **logistic regression** example, where some preprocessing (one hot encoding) +is conducted on the target column to show you more the capabilities of igel. + +Finally, the multioutput-example contains a **multioutput regression** example. + +I suggest you play around with the examples and igel cli. However, +you can also directly execute the fit.py, evaluate.py and predict.py if you want to. + +Model A/B Testing +---------------- + +The ``igel`` package includes functionality to compare two trained models using statistical tests. +This is useful when you want to determine if there are significant differences between model versions. + +Command Line Usage +~~~~~~~~~~~~~~~~ + +To compare two trained models:: + + python -m igel compare-models \ + --model_a_path path/to/model_a.joblib \ + --model_b_path path/to/model_b.joblib \ + --test_data path/to/test_data.csv \ + --problem_type classification + +Options: + - ``--model_a_path``, ``-ma``: Path to first model + - ``--model_b_path``, ``-mb``: Path to second model + - ``--test_data``, ``-td``: Path to test dataset + - ``--problem_type``, ``-pt``: Type of problem (classification or regression) + +Python API Usage +~~~~~~~~~~~~~~ + +You can also use the A/B testing functionality directly in your Python code:: + + from igel.ab_testing import ModelComparison + + # Initialize comparison + comparison = ModelComparison(model_a, model_b, test_type="classification") + + # Compare models + results = comparison.compare_predictions(X_test, y_test) + + # Generate report + report = comparison.generate_report(results) + print(report) + +The comparison includes: + - Performance metrics (accuracy for classification, MSE/R² for regression) + - Statistical significance tests + - Detailed comparison report + +Statistical Tests +~~~~~~~~~~~~~~~ + +- For classification problems: McNemar's test +- For regression problems: Wilcoxon signed-rank test + +The p-value indicates whether the difference between models is statistically significant (p < 0.05). diff --git a/igel/examples/auto-ml/igel.yaml b/igel/examples/auto-ml/igel.yaml new file mode 100644 index 0000000..6a9e804 --- /dev/null +++ b/igel/examples/auto-ml/igel.yaml @@ -0,0 +1,9 @@ + + +model: + type: ImageClassification + arguments: + max_trials: 1 + +training: + epochs: 10 diff --git a/igel/examples/clustering/fit.py b/igel/examples/clustering/fit.py new file mode 100644 index 0000000..ee9cd74 --- /dev/null +++ b/igel/examples/clustering/fit.py @@ -0,0 +1,24 @@ +from igel import Igel + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel fit -dp path_to_dataset -yml path_to_yaml_file`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + +=============================================================================================================== + +This example fits a machine learning model on the indian-diabetes dataset + +- default model here is the neural network and the configuration are provided in neural-network.yaml file +- You can switch to random forest by providing the random-forest.yaml as the config file in the parameters + +""" + +mock_fit_params = {'data_path': '../data/clustering-data/train.csv', + 'yaml_path': './igel.yaml', + 'cmd': 'fit'} + +Igel(**mock_fit_params) + diff --git a/igel/examples/clustering/igel.yaml b/igel/examples/clustering/igel.yaml new file mode 100644 index 0000000..4e8afb2 --- /dev/null +++ b/igel/examples/clustering/igel.yaml @@ -0,0 +1,18 @@ + +dataset: + type: csv + +# model definition +model: + type: clustering + algorithm: KMeans + arguments: # if you don't provide these arguments, then default will be used + n_clusters: 3 + init: random + n_init: 10 + max_iter: 300 + tol: 0.0004 + random_state: 0 + +# target you want to predict +target: # you can keep this empty since you want to use clustering diff --git a/igel/examples/clustering/predict.py b/igel/examples/clustering/predict.py new file mode 100644 index 0000000..afa6d0f --- /dev/null +++ b/igel/examples/clustering/predict.py @@ -0,0 +1,21 @@ +from igel import Igel + + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel predict -dp path_to_dataset`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + + +=============================================================================================================== + +This example uses the pre-fitted machine learning model to generate predictions + +""" + +mock_pred_params = {'data_path': '../data/clustering-data/train.csv', + 'cmd': 'predict'} + +Igel(**mock_pred_params) diff --git a/igel/examples/clustering_config.yaml b/igel/examples/clustering_config.yaml new file mode 100644 index 0000000..08c3cdd --- /dev/null +++ b/igel/examples/clustering_config.yaml @@ -0,0 +1,8 @@ +input: data/clustering_data.csv +# No target for clustering +model: + type: clustering + algorithm: KMeans + params: + n_clusters: 3 + random_state: 42 diff --git a/igel/examples/cv-example/evaluate.py b/igel/examples/cv-example/evaluate.py new file mode 100644 index 0000000..1a00199 --- /dev/null +++ b/igel/examples/cv-example/evaluate.py @@ -0,0 +1,21 @@ +from igel import Igel + + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel evaluate -dp path_to_dataset`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + + +=============================================================================================================== + +This example uses the pre-fitted machine learning model to evaluate its performance + +""" + +mock_eval_params = {'data_path': '../data/indian-diabetes/eval-indians-diabetes.csv', + 'cmd': 'evaluate'} + +Igel(**mock_eval_params) diff --git a/igel/examples/cv-example/fit.py b/igel/examples/cv-example/fit.py new file mode 100644 index 0000000..f4acf46 --- /dev/null +++ b/igel/examples/cv-example/fit.py @@ -0,0 +1,24 @@ +from igel import Igel + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel fit -dp path_to_dataset -yml path_to_yaml_file`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + +=============================================================================================================== + +This example fits a machine learning model on the indian-diabetes dataset + +- default model here is the neural network and the configuration are provided in neural-network.yaml file +- You can switch to random forest by providing the random-forest.yaml as the config file in the parameters + +""" + +mock_fit_params = {'data_path': '../data/indian-diabetes/train-indians-diabetes.csv', + 'yaml_path': './igel.yaml', + 'cmd': 'fit'} + +Igel(**mock_fit_params) + diff --git a/igel/examples/cv-example/igel.json b/igel/examples/cv-example/igel.json new file mode 100644 index 0000000..fd2956e --- /dev/null +++ b/igel/examples/cv-example/igel.json @@ -0,0 +1,22 @@ +{ + "dataset": { + "split": { + "test_size": 0.2 + }, + + "preprocess": { + "scale": { + "method": "standard", + "target": "inputs" + } + } + }, + + "model": { + "type": "classification", + "algorithm": "Ridge", + "use_cv_estimator": true + }, + + "target": ["sick"] +} diff --git a/igel/examples/cv-example/igel.yaml b/igel/examples/cv-example/igel.yaml new file mode 100644 index 0000000..c4d45d0 --- /dev/null +++ b/igel/examples/cv-example/igel.yaml @@ -0,0 +1,26 @@ +# dataset operations +dataset: + type: csv + split: # split options + test_size: 0.2 # 0.2 means 20% for the test data, so 80% are automatically for training + shuffle: True # whether to shuffle the data before/while splitting + + preprocess: + scale: # scaling options + method: standard # standardization will scale values to have a 0 mean and 1 standard deviation | you can also try minmax + target: inputs # scale inputs. | other possible values: [outputs, all] # if you choose all then all values in the dataset will be scaled + + +# model definition +model: + type: classification + algorithm: Ridge + use_cv_estimator: true + cross_validate: + cv: 3 + n_jobs: 1 + verbose: 1 + +# target you want to predict +target: + - sick diff --git a/igel/examples/cv-example/predict.py b/igel/examples/cv-example/predict.py new file mode 100644 index 0000000..68bef1d --- /dev/null +++ b/igel/examples/cv-example/predict.py @@ -0,0 +1,21 @@ +from igel import Igel + + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel predict -dp path_to_dataset`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + + +=============================================================================================================== + +This example uses the pre-fitted machine learning model to generate predictions + +""" + +mock_pred_params = {'data_path': '../data/indian-diabetes/test-indians-diabetes.csv', + 'cmd': 'predict'} + +Igel(**mock_pred_params) diff --git a/igel/examples/data/clustering-data/raw_data.csv b/igel/examples/data/clustering-data/raw_data.csv new file mode 100644 index 0000000..47af889 --- /dev/null +++ b/igel/examples/data/clustering-data/raw_data.csv @@ -0,0 +1,151 @@ +feature1,feature2,result +2.6050973193764344,1.2252955252992357,1 +0.5323772047314387,3.3133890933364265,0 +0.8023140038834188,4.38196181200038,0 +0.5285367979496572,4.497238576378021,0 +2.6185854824861314,0.3576979057562252,1 +1.5914154189103553,4.904977251840595,0 +1.742659685725724,5.038466712398533,0 +2.375333284481674,0.08918563778251964,1 +-2.1213336421139197,2.664474084183779,2 +1.720396175444295,5.251731915463681,0 +3.136885496073223,1.5659276346561328,1 +-0.3749456643799345,2.387874349972349,2 +-1.84562252599802,2.7192463541687237,2 +0.7214439876706683,4.084750176642797,0 +0.16117090506347276,4.535178455211277,0 +-1.9991271411810305,2.7128574147318485,2 +-1.4780415293644775,3.2093591012097695,2 +1.8706766024616561,0.7779740711499736,1 +-1.5933443020153832,2.7689868216322586,2 +2.03562611231913,0.31361691106733813,1 +0.6400398546585195,4.1240107466781195,0 +2.441162797039712,1.3094157369198025,1 +1.1328039293719456,3.8767394577975276,0 +1.048291864126934,5.030924080929878,0 +-1.2663715749955262,2.6299882764265896,2 +2.316905851698755,0.8118904943268128,1 +2.362307206605918,1.3587669957212003,1 +1.209101298411725,3.535665484309778,0 +-2.5422462471126526,3.9501286920127825,2 +1.4815331952273267,0.6787536375657198,1 +-1.5948788635610658,3.48632794263447,2 +-1.8255620477045866,2.7989213964651194,2 +-1.1337400321674174,2.6846727129651513,2 +-1.7587020005449525,3.1586229982198537,2 +0.34987239852153196,4.692532505364345,0 +1.6854860232372213,1.6691709576413047,1 +2.989047001646163,1.3506859890756295,1 +1.7373444822434374,1.2358803074111866,1 +0.6591090317060133,4.122416744454821,0 +1.154453277133696,4.657073911544364,0 +-1.3273808404018403,1.5315858831197977,2 +-1.6814104977454665,2.0798803581681344,2 +0.34102757930362826,4.788485681527396,0 +1.878270565806136,0.21018801322892744,1 +2.1386042691191425,1.21517937838399,1 +2.4836828273842233,0.5721508632878634,1 +-1.1811346376868121,3.265256833161126,2 +2.1111473905402987,3.5766044901490077,0 +-1.1937124722059482,2.687522367638846,2 +1.4513142873092897,4.228108723299541,0 +1.8376907455720592,1.8222955241776078,1 +0.4408937677912238,4.831013190913958,0 +1.0804075675849252,4.792106845690246,0 +1.8484580310530043,0.523936254217558,1 +2.391414899939096,1.1013945780584922,1 +-1.4486507442392176,3.0339727794332605,2 +0.7208675097620585,3.7134712353871837,0 +3.0167385346730704,1.637921055655149,1 +-1.1819949309545112,3.568805376115622,2 +1.3408153596352634,4.368278782827096,0 +-2.3183732118894462,3.2230719508867254,2 +-0.5489478590965583,3.112928922677086,2 +-1.6823470990711413,2.9665823444675143,2 +-1.5354142201526257,3.1074581291106638,2 +1.0649831496733715,4.102896859344259,0 +-0.3972495378764793,2.8967536855028015,2 +1.039726124898305,4.504782009170741,0 +1.6246546789055267,1.8526961364874537,1 +-0.3002248293705443,4.63059662516857,0 +0.12313498323398875,5.279175025064285,0 +1.5459704208181453,3.686374417271564,0 +1.4425497620177938,1.3198451481387103,1 +2.528893505319752,0.8201586133925197,1 +0.3897083759894151,5.275597920273036,0 +1.5381461005406456,1.2384609190787503,1 +0.8204938124828087,4.3318699985632625,0 +1.5656598641263204,4.213824909542215,0 +-1.9335861428240069,2.184670097431866,2 +-1.3837321687757655,3.222304178570848,2 +0.9621789643771677,4.517953262713599,0 +1.71810119110419,0.9135789390751125,1 +1.6535626893459574,0.5528887710628365,1 +0.45199359601294875,3.5937783588589025,0 +1.1982016949192078,4.470624491135523,0 +2.204386608535906,1.5608566082814521,1 +3.2468399088648487,1.3699034034331437,1 +2.5156969333232873,1.0570274864094475,1 +-1.7983347512302643,3.1259072844638354,2 +-2.049530696295383,3.5234549061744733,2 +2.3678832469464717,0.09663483213456303,1 +2.2434802870106925,0.3479632646458396,1 +0.9991493371972181,4.210195402435473,0 +1.3096387250800752,1.1173595105702052,1 +0.7746816050599052,4.915009862639604,0 +1.7079835915671953,0.8228463897741014,1 +1.9178454270746221,3.6299077968771667,0 +-2.004876513468294,2.7448913734834295,2 +-2.1049952291328617,3.3084813121881194,2 +1.3973138161771175,0.6668713575305824,1 +2.0211467187705625,1.7543350207626203,1 +1.6703094842102197,1.1672882555838455,1 +2.5299779248957583,0.9414392806305323,1 +-2.180167439089564,3.746947601142217,2 +2.006041259220162,0.5659245167568832,1 +1.5030758517186527,0.9237461995684241,1 +1.0537437913949532,4.492868587249477,0 +-1.726628527853343,3.102910205256882,2 +1.7233096151252982,4.201208195565489,0 +0.9246606526497161,4.509086578417576,0 +0.3936951581548168,4.754200570925484,0 +-1.313774647832855,3.2563362788482455,2 +0.7826066698425189,4.152635952160722,0 +1.8275012696742063,0.9064032394504652,1 +-1.2649585013048026,2.9620933048554536,2 +0.9815200889069053,5.196722574401307,0 +-2.4950439161451587,3.012271559730037,2 +1.009528689738079,4.455023276318281,0 +1.408488177976248,3.932704817245169,0 +-1.280033124547311,2.8598302918159395,2 +-1.8250610324471594,2.8915986131983495,2 +0.5408715039555542,4.014362495066182,0 +2.6492824176090286,1.0561349659003616,1 +0.522620896354874,4.329760025346459,0 +0.1693211547675193,4.197417187341405,0 +1.8062512960867254,1.8624296868464296,1 +1.9212658359571877,1.2988918578361344,1 +-1.539067075421373,2.5488668067302784,2 +1.682890110408658,0.48444439060842964,1 +-2.2973025204442177,2.9495132584332886,2 +-1.4559274315615156,2.7582180527605753,2 +-1.3869417137002862,2.8688070665116436,2 +-1.0718145591756094,3.0764913689287736,2 +1.4088390665711537,1.0311890946125284,1 +-1.5859860357861904,2.5777931593346977,2 +-1.5821743418381418,3.4279686171910226,2 +-0.779661740976221,1.8828897488263565,2 +0.569696937524268,3.4406460262825513,0 +-1.8531083044154153,2.722405573739322,2 +1.5988564087107986,1.4561718039858633,1 +-1.840947793042695,2.677368702102729,2 +1.3567889411199918,4.364624835694804,0 +1.1774408991352696,3.961382281978233,0 +1.7334583200164326,-0.21403791617427648,1 +2.343562929740348,0.7935142821489394,1 +-0.9507382308303454,3.4576915573515334,2 +-2.2389344677131713,2.671222319652026,2 +-1.872928937114101,3.6860707884560218,2 +-1.8897027024536976,2.226200283635595,2 +2.2532708777637005,0.35113290557268395,1 +1.5551598477380955,0.12527811154913104,1 diff --git a/igel/examples/data/clustering-data/train.csv b/igel/examples/data/clustering-data/train.csv new file mode 100644 index 0000000..5bdfe98 --- /dev/null +++ b/igel/examples/data/clustering-data/train.csv @@ -0,0 +1,151 @@ +feature1,feature2 +2.6050973193764344,1.2252955252992357 +0.5323772047314387,3.3133890933364265 +0.8023140038834188,4.38196181200038 +0.5285367979496572,4.497238576378021 +2.6185854824861314,0.3576979057562252 +1.5914154189103553,4.904977251840595 +1.742659685725724,5.038466712398533 +2.375333284481674,0.08918563778251964 +-2.1213336421139197,2.664474084183779 +1.720396175444295,5.251731915463681 +3.136885496073223,1.5659276346561328 +-0.3749456643799345,2.387874349972349 +-1.84562252599802,2.7192463541687237 +0.7214439876706683,4.084750176642797 +0.16117090506347276,4.535178455211277 +-1.9991271411810305,2.7128574147318485 +-1.4780415293644775,3.2093591012097695 +1.8706766024616561,0.7779740711499736 +-1.5933443020153832,2.7689868216322586 +2.03562611231913,0.31361691106733813 +0.6400398546585195,4.1240107466781195 +2.441162797039712,1.3094157369198025 +1.1328039293719456,3.8767394577975276 +1.048291864126934,5.030924080929878 +-1.2663715749955262,2.6299882764265896 +2.316905851698755,0.8118904943268128 +2.362307206605918,1.3587669957212003 +1.209101298411725,3.535665484309778 +-2.5422462471126526,3.9501286920127825 +1.4815331952273267,0.6787536375657198 +-1.5948788635610658,3.48632794263447 +-1.8255620477045866,2.7989213964651194 +-1.1337400321674174,2.6846727129651513 +-1.7587020005449525,3.1586229982198537 +0.34987239852153196,4.692532505364345 +1.6854860232372213,1.6691709576413047 +2.989047001646163,1.3506859890756295 +1.7373444822434374,1.2358803074111866 +0.6591090317060133,4.122416744454821 +1.154453277133696,4.657073911544364 +-1.3273808404018403,1.5315858831197977 +-1.6814104977454665,2.0798803581681344 +0.34102757930362826,4.788485681527396 +1.878270565806136,0.21018801322892744 +2.1386042691191425,1.21517937838399 +2.4836828273842233,0.5721508632878634 +-1.1811346376868121,3.265256833161126 +2.1111473905402987,3.5766044901490077 +-1.1937124722059482,2.687522367638846 +1.4513142873092897,4.228108723299541 +1.8376907455720592,1.8222955241776078 +0.4408937677912238,4.831013190913958 +1.0804075675849252,4.792106845690246 +1.8484580310530043,0.523936254217558 +2.391414899939096,1.1013945780584922 +-1.4486507442392176,3.0339727794332605 +0.7208675097620585,3.7134712353871837 +3.0167385346730704,1.637921055655149 +-1.1819949309545112,3.568805376115622 +1.3408153596352634,4.368278782827096 +-2.3183732118894462,3.2230719508867254 +-0.5489478590965583,3.112928922677086 +-1.6823470990711413,2.9665823444675143 +-1.5354142201526257,3.1074581291106638 +1.0649831496733715,4.102896859344259 +-0.3972495378764793,2.8967536855028015 +1.039726124898305,4.504782009170741 +1.6246546789055267,1.8526961364874537 +-0.3002248293705443,4.63059662516857 +0.12313498323398875,5.279175025064285 +1.5459704208181453,3.686374417271564 +1.4425497620177938,1.3198451481387103 +2.528893505319752,0.8201586133925197 +0.3897083759894151,5.275597920273036 +1.5381461005406456,1.2384609190787503 +0.8204938124828087,4.3318699985632625 +1.5656598641263204,4.213824909542215 +-1.9335861428240069,2.184670097431866 +-1.3837321687757655,3.222304178570848 +0.9621789643771677,4.517953262713599 +1.71810119110419,0.9135789390751125 +1.6535626893459574,0.5528887710628365 +0.45199359601294875,3.5937783588589025 +1.1982016949192078,4.470624491135523 +2.204386608535906,1.5608566082814521 +3.2468399088648487,1.3699034034331437 +2.5156969333232873,1.0570274864094475 +-1.7983347512302643,3.1259072844638354 +-2.049530696295383,3.5234549061744733 +2.3678832469464717,0.09663483213456303 +2.2434802870106925,0.3479632646458396 +0.9991493371972181,4.210195402435473 +1.3096387250800752,1.1173595105702052 +0.7746816050599052,4.915009862639604 +1.7079835915671953,0.8228463897741014 +1.9178454270746221,3.6299077968771667 +-2.004876513468294,2.7448913734834295 +-2.1049952291328617,3.3084813121881194 +1.3973138161771175,0.6668713575305824 +2.0211467187705625,1.7543350207626203 +1.6703094842102197,1.1672882555838455 +2.5299779248957583,0.9414392806305323 +-2.180167439089564,3.746947601142217 +2.006041259220162,0.5659245167568832 +1.5030758517186527,0.9237461995684241 +1.0537437913949532,4.492868587249477 +-1.726628527853343,3.102910205256882 +1.7233096151252982,4.201208195565489 +0.9246606526497161,4.509086578417576 +0.3936951581548168,4.754200570925484 +-1.313774647832855,3.2563362788482455 +0.7826066698425189,4.152635952160722 +1.8275012696742063,0.9064032394504652 +-1.2649585013048026,2.9620933048554536 +0.9815200889069053,5.196722574401307 +-2.4950439161451587,3.012271559730037 +1.009528689738079,4.455023276318281 +1.408488177976248,3.932704817245169 +-1.280033124547311,2.8598302918159395 +-1.8250610324471594,2.8915986131983495 +0.5408715039555542,4.014362495066182 +2.6492824176090286,1.0561349659003616 +0.522620896354874,4.329760025346459 +0.1693211547675193,4.197417187341405 +1.8062512960867254,1.8624296868464296 +1.9212658359571877,1.2988918578361344 +-1.539067075421373,2.5488668067302784 +1.682890110408658,0.48444439060842964 +-2.2973025204442177,2.9495132584332886 +-1.4559274315615156,2.7582180527605753 +-1.3869417137002862,2.8688070665116436 +-1.0718145591756094,3.0764913689287736 +1.4088390665711537,1.0311890946125284 +-1.5859860357861904,2.5777931593346977 +-1.5821743418381418,3.4279686171910226 +-0.779661740976221,1.8828897488263565 +0.569696937524268,3.4406460262825513 +-1.8531083044154153,2.722405573739322 +1.5988564087107986,1.4561718039858633 +-1.840947793042695,2.677368702102729 +1.3567889411199918,4.364624835694804 +1.1774408991352696,3.961382281978233 +1.7334583200164326,-0.21403791617427648 +2.343562929740348,0.7935142821489394 +-0.9507382308303454,3.4576915573515334 +-2.2389344677131713,2.671222319652026 +-1.872928937114101,3.6860707884560218 +-1.8897027024536976,2.226200283635595 +2.2532708777637005,0.35113290557268395 +1.5551598477380955,0.12527811154913104 diff --git a/igel/examples/data/indian-diabetes/eval-indians-diabetes.csv b/igel/examples/data/indian-diabetes/eval-indians-diabetes.csv new file mode 100644 index 0000000..97a1d28 --- /dev/null +++ b/igel/examples/data/indian-diabetes/eval-indians-diabetes.csv @@ -0,0 +1,100 @@ +n_pregnant,plasma_concentration,blood_pressure,TST,insulin,BMI,DPF,age,sick +6,148,72,35,0,33.6,627,50,1 +1,85,66,29,0,26.6,351,31,0 +8,183,64,0,0,23.3,672,32,1 +1,89,66,23,94,28.1,167,21,0 +0,137,40,35,168,43.1,2288,33,1 +5,116,74,0,0,25.6,201,30,0 +3,78,50,32,88,31.0,248,26,1 +10,115,0,0,0,35.3,134,29,0 +2,197,70,45,543,30.5,158,53,1 +8,125,96,0,0,0.0,232,54,1 +4,110,92,0,0,37.6,191,30,0 +10,168,74,0,0,38.0,537,34,1 +10,139,80,0,0,27.1,1441,57,0 +1,189,60,23,846,30.1,398,59,1 +5,166,72,19,175,25.8,587,51,1 +7,100,0,0,0,30.0,484,32,1 +0,118,84,47,230,45.8,551,31,1 +7,107,74,0,0,29.6,254,31,1 +1,103,30,38,83,43.3,183,33,0 +1,115,70,30,96,34.6,529,32,1 +3,126,88,41,235,39.3,704,27,0 +8,99,84,0,0,35.4,388,50,0 +7,196,90,0,0,39.8,451,41,1 +9,119,80,35,0,29.0,263,29,1 +11,143,94,33,146,36.6,254,51,1 +10,125,70,26,115,31.1,205,41,1 +7,147,76,0,0,39.4,257,43,1 +1,97,66,15,140,23.2,487,22,0 +13,145,82,19,110,22.2,245,57,0 +5,117,92,0,0,34.1,337,38,0 +5,109,75,26,0,36.0,546,60,0 +3,158,76,36,245,31.6,851,28,1 +3,88,58,11,54,24.8,267,22,0 +6,92,92,0,0,19.9,188,28,0 +10,122,78,31,0,27.6,512,45,0 +4,103,60,33,192,24.0,966,33,0 +11,138,76,0,0,33.2,420,35,0 +9,102,76,37,0,32.9,665,46,1 +2,90,68,42,0,38.2,503,27,1 +4,111,72,47,207,37.1,1390,56,1 +3,180,64,25,70,34.0,271,26,0 +7,133,84,0,0,40.2,696,37,0 +7,106,92,18,0,22.7,235,48,0 +9,171,110,24,240,45.4,721,54,1 +7,159,64,0,0,27.4,294,40,0 +0,180,66,39,0,42.0,1893,25,1 +1,146,56,0,0,29.7,564,29,0 +2,71,70,27,0,28.0,586,22,0 +7,103,66,32,0,39.1,344,31,1 +7,105,0,0,0,0.0,305,24,0 +1,103,80,11,82,19.4,491,22,0 +1,101,50,15,36,24.2,526,26,0 +5,88,66,21,23,24.4,342,30,0 +8,176,90,34,300,33.7,467,58,1 +7,150,66,42,342,34.7,718,42,0 +1,73,50,10,0,23.0,248,21,0 +7,187,68,39,304,37.7,254,41,1 +0,100,88,60,110,46.8,962,31,0 +0,146,82,0,0,40.5,1781,44,0 +0,105,64,41,142,41.5,173,22,0 +2,84,0,0,0,0.0,304,21,0 +8,133,72,0,0,32.9,270,39,1 +5,44,62,0,0,25.0,587,36,0 +2,141,58,34,128,25.4,699,24,0 +7,114,66,0,0,32.8,258,42,1 +5,99,74,27,0,29.0,203,32,0 +0,109,88,30,0,32.5,855,38,1 +2,109,92,0,0,42.7,845,54,0 +1,95,66,13,38,19.6,334,25,0 +4,146,85,27,100,28.9,189,27,0 +2,100,66,20,90,32.9,867,28,1 +5,139,64,35,140,28.6,411,26,0 +13,126,90,0,0,43.4,583,42,1 +4,129,86,20,270,35.1,231,23,0 +1,79,75,30,0,32.0,396,22,0 +1,0,48,20,0,24.7,140,22,0 +7,62,78,0,0,32.6,391,41,0 +5,95,72,33,0,37.7,370,27,0 +0,131,0,0,0,43.2,270,26,1 +2,112,66,22,0,25.0,307,24,0 +3,113,44,13,0,22.4,140,22,0 +2,74,0,0,0,0.0,102,22,0 +7,83,78,26,71,29.3,767,36,0 +0,101,65,28,0,24.6,237,22,0 +5,137,108,0,0,48.8,227,37,1 +2,110,74,29,125,32.4,698,27,0 +13,106,72,54,0,36.6,178,45,0 +2,100,68,25,71,38.5,324,26,0 +15,136,70,32,110,37.1,153,43,1 +1,107,68,19,0,26.5,165,24,0 +1,80,55,0,0,19.1,258,21,0 +4,123,80,15,176,32.0,443,34,0 +7,81,78,40,48,46.7,261,42,0 +4,134,72,0,0,23.8,277,60,1 +2,142,82,18,64,24.7,761,21,0 +6,144,72,27,228,33.9,255,40,0 +2,92,62,28,0,31.6,130,24,0 +1,71,48,18,76,20.4,323,22,0 +6,93,50,30,64,28.7,356,23,0 diff --git a/igel/examples/data/indian-diabetes/test-indians-diabetes.csv b/igel/examples/data/indian-diabetes/test-indians-diabetes.csv new file mode 100644 index 0000000..edb3648 --- /dev/null +++ b/igel/examples/data/indian-diabetes/test-indians-diabetes.csv @@ -0,0 +1,22 @@ +n_pregnant,plasma_concentration,blood_pressure,TST,insulin,BMI,DPF,age +6,148,72,35,0,33.6,627,50 +1,85,66,29,0,26.6,351,31 +8,183,64,0,0,23.3,672,32 +1,89,66,23,94,28.1,167,21 +0,137,40,35,168,43.1,2288,33 +5,116,74,0,0,25.6,201,30 +3,78,50,32,88,31.0,248,26 +10,115,0,0,0,35.3,134,29 +2,197,70,45,543,30.5,158,53 +8,125,96,0,0,0.0,232,54 +4,110,92,0,0,37.6,191,30 +10,168,74,0,0,38.0,537,34 +10,139,80,0,0,27.1,1441,57 +1,189,60,23,846,30.1,398,59 +5,166,72,19,175,25.8,587,51 +7,100,0,0,0,30.0,484,32 +0,118,84,47,230,45.8,551,31 +7,107,74,0,0,29.6,254,31 +1,103,30,38,83,43.3,183,33 +1,115,70,30,96,34.6,529,32 +3,126,88,41,235,39.3,704,27 diff --git a/igel/examples/data/indian-diabetes/train-indians-diabetes.csv b/igel/examples/data/indian-diabetes/train-indians-diabetes.csv new file mode 100644 index 0000000..7602de4 --- /dev/null +++ b/igel/examples/data/indian-diabetes/train-indians-diabetes.csv @@ -0,0 +1,769 @@ +n_pregnant,plasma_concentration,blood_pressure,TST,insulin,BMI,DPF,age,sick +6,148,72,35,0,33.6,627,50,1 +1,85,66,29,0,26.6,351,31,0 +8,183,64,0,0,23.3,672,32,1 +1,89,66,23,94,28.1,167,21,0 +0,137,40,35,168,43.1,2288,33,1 +5,116,74,0,0,25.6,201,30,0 +3,78,50,32,88,31.0,248,26,1 +10,115,0,0,0,35.3,134,29,0 +2,197,70,45,543,30.5,158,53,1 +8,125,96,0,0,0.0,232,54,1 +4,110,92,0,0,37.6,191,30,0 +10,168,74,0,0,38.0,537,34,1 +10,139,80,0,0,27.1,1441,57,0 +1,189,60,23,846,30.1,398,59,1 +5,166,72,19,175,25.8,587,51,1 +7,100,0,0,0,30.0,484,32,1 +0,118,84,47,230,45.8,551,31,1 +7,107,74,0,0,29.6,254,31,1 +1,103,30,38,83,43.3,183,33,0 +1,115,70,30,96,34.6,529,32,1 +3,126,88,41,235,39.3,704,27,0 +8,99,84,0,0,35.4,388,50,0 +7,196,90,0,0,39.8,451,41,1 +9,119,80,35,0,29.0,263,29,1 +11,143,94,33,146,36.6,254,51,1 +10,125,70,26,115,31.1,205,41,1 +7,147,76,0,0,39.4,257,43,1 +1,97,66,15,140,23.2,487,22,0 +13,145,82,19,110,22.2,245,57,0 +5,117,92,0,0,34.1,337,38,0 +5,109,75,26,0,36.0,546,60,0 +3,158,76,36,245,31.6,851,28,1 +3,88,58,11,54,24.8,267,22,0 +6,92,92,0,0,19.9,188,28,0 +10,122,78,31,0,27.6,512,45,0 +4,103,60,33,192,24.0,966,33,0 +11,138,76,0,0,33.2,420,35,0 +9,102,76,37,0,32.9,665,46,1 +2,90,68,42,0,38.2,503,27,1 +4,111,72,47,207,37.1,1390,56,1 +3,180,64,25,70,34.0,271,26,0 +7,133,84,0,0,40.2,696,37,0 +7,106,92,18,0,22.7,235,48,0 +9,171,110,24,240,45.4,721,54,1 +7,159,64,0,0,27.4,294,40,0 +0,180,66,39,0,42.0,1893,25,1 +1,146,56,0,0,29.7,564,29,0 +2,71,70,27,0,28.0,586,22,0 +7,103,66,32,0,39.1,344,31,1 +7,105,0,0,0,0.0,305,24,0 +1,103,80,11,82,19.4,491,22,0 +1,101,50,15,36,24.2,526,26,0 +5,88,66,21,23,24.4,342,30,0 +8,176,90,34,300,33.7,467,58,1 +7,150,66,42,342,34.7,718,42,0 +1,73,50,10,0,23.0,248,21,0 +7,187,68,39,304,37.7,254,41,1 +0,100,88,60,110,46.8,962,31,0 +0,146,82,0,0,40.5,1781,44,0 +0,105,64,41,142,41.5,173,22,0 +2,84,0,0,0,0.0,304,21,0 +8,133,72,0,0,32.9,270,39,1 +5,44,62,0,0,25.0,587,36,0 +2,141,58,34,128,25.4,699,24,0 +7,114,66,0,0,32.8,258,42,1 +5,99,74,27,0,29.0,203,32,0 +0,109,88,30,0,32.5,855,38,1 +2,109,92,0,0,42.7,845,54,0 +1,95,66,13,38,19.6,334,25,0 +4,146,85,27,100,28.9,189,27,0 +2,100,66,20,90,32.9,867,28,1 +5,139,64,35,140,28.6,411,26,0 +13,126,90,0,0,43.4,583,42,1 +4,129,86,20,270,35.1,231,23,0 +1,79,75,30,0,32.0,396,22,0 +1,0,48,20,0,24.7,140,22,0 +7,62,78,0,0,32.6,391,41,0 +5,95,72,33,0,37.7,370,27,0 +0,131,0,0,0,43.2,270,26,1 +2,112,66,22,0,25.0,307,24,0 +3,113,44,13,0,22.4,140,22,0 +2,74,0,0,0,0.0,102,22,0 +7,83,78,26,71,29.3,767,36,0 +0,101,65,28,0,24.6,237,22,0 +5,137,108,0,0,48.8,227,37,1 +2,110,74,29,125,32.4,698,27,0 +13,106,72,54,0,36.6,178,45,0 +2,100,68,25,71,38.5,324,26,0 +15,136,70,32,110,37.1,153,43,1 +1,107,68,19,0,26.5,165,24,0 +1,80,55,0,0,19.1,258,21,0 +4,123,80,15,176,32.0,443,34,0 +7,81,78,40,48,46.7,261,42,0 +4,134,72,0,0,23.8,277,60,1 +2,142,82,18,64,24.7,761,21,0 +6,144,72,27,228,33.9,255,40,0 +2,92,62,28,0,31.6,130,24,0 +1,71,48,18,76,20.4,323,22,0 +6,93,50,30,64,28.7,356,23,0 +1,122,90,51,220,49.7,325,31,1 +1,163,72,0,0,39.0,1222,33,1 +1,151,60,0,0,26.1,179,22,0 +0,125,96,0,0,22.5,262,21,0 +1,81,72,18,40,26.6,283,24,0 +2,85,65,0,0,39.6,930,27,0 +1,126,56,29,152,28.7,801,21,0 +1,96,122,0,0,22.4,207,27,0 +4,144,58,28,140,29.5,287,37,0 +3,83,58,31,18,34.3,336,25,0 +0,95,85,25,36,37.4,247,24,1 +3,171,72,33,135,33.3,199,24,1 +8,155,62,26,495,34.0,543,46,1 +1,89,76,34,37,31.2,192,23,0 +4,76,62,0,0,34.0,391,25,0 +7,160,54,32,175,30.5,588,39,1 +4,146,92,0,0,31.2,539,61,1 +5,124,74,0,0,34.0,220,38,1 +5,78,48,0,0,33.7,654,25,0 +4,97,60,23,0,28.2,443,22,0 +4,99,76,15,51,23.2,223,21,0 +0,162,76,56,100,53.2,759,25,1 +6,111,64,39,0,34.2,260,24,0 +2,107,74,30,100,33.6,404,23,0 +5,132,80,0,0,26.8,186,69,0 +0,113,76,0,0,33.3,278,23,1 +1,88,30,42,99,55.0,496,26,1 +3,120,70,30,135,42.9,452,30,0 +1,118,58,36,94,33.3,261,23,0 +1,117,88,24,145,34.5,403,40,1 +0,105,84,0,0,27.9,741,62,1 +4,173,70,14,168,29.7,361,33,1 +9,122,56,0,0,33.3,1114,33,1 +3,170,64,37,225,34.5,356,30,1 +8,84,74,31,0,38.3,457,39,0 +2,96,68,13,49,21.1,647,26,0 +2,125,60,20,140,33.8,88,31,0 +0,100,70,26,50,30.8,597,21,0 +0,93,60,25,92,28.7,532,22,0 +0,129,80,0,0,31.2,703,29,0 +5,105,72,29,325,36.9,159,28,0 +3,128,78,0,0,21.1,268,55,0 +5,106,82,30,0,39.5,286,38,0 +2,108,52,26,63,32.5,318,22,0 +10,108,66,0,0,32.4,272,42,1 +4,154,62,31,284,32.8,237,23,0 +0,102,75,23,0,0.0,572,21,0 +9,57,80,37,0,32.8,96,41,0 +2,106,64,35,119,30.5,1400,34,0 +5,147,78,0,0,33.7,218,65,0 +2,90,70,17,0,27.3,85,22,0 +1,136,74,50,204,37.4,399,24,0 +4,114,65,0,0,21.9,432,37,0 +9,156,86,28,155,34.3,1189,42,1 +1,153,82,42,485,40.6,687,23,0 +8,188,78,0,0,47.9,137,43,1 +7,152,88,44,0,50.0,337,36,1 +2,99,52,15,94,24.6,637,21,0 +1,109,56,21,135,25.2,833,23,0 +2,88,74,19,53,29.0,229,22,0 +17,163,72,41,114,40.9,817,47,1 +4,151,90,38,0,29.7,294,36,0 +7,102,74,40,105,37.2,204,45,0 +0,114,80,34,285,44.2,167,27,0 +2,100,64,23,0,29.7,368,21,0 +0,131,88,0,0,31.6,743,32,1 +6,104,74,18,156,29.9,722,41,1 +3,148,66,25,0,32.5,256,22,0 +4,120,68,0,0,29.6,709,34,0 +4,110,66,0,0,31.9,471,29,0 +3,111,90,12,78,28.4,495,29,0 +6,102,82,0,0,30.8,180,36,1 +6,134,70,23,130,35.4,542,29,1 +2,87,0,23,0,28.9,773,25,0 +1,79,60,42,48,43.5,678,23,0 +2,75,64,24,55,29.7,370,33,0 +8,179,72,42,130,32.7,719,36,1 +6,85,78,0,0,31.2,382,42,0 +0,129,110,46,130,67.1,319,26,1 +5,143,78,0,0,45.0,190,47,0 +5,130,82,0,0,39.1,956,37,1 +6,87,80,0,0,23.2,84,32,0 +0,119,64,18,92,34.9,725,23,0 +1,0,74,20,23,27.7,299,21,0 +5,73,60,0,0,26.8,268,27,0 +4,141,74,0,0,27.6,244,40,0 +7,194,68,28,0,35.9,745,41,1 +8,181,68,36,495,30.1,615,60,1 +1,128,98,41,58,32.0,1321,33,1 +8,109,76,39,114,27.9,640,31,1 +5,139,80,35,160,31.6,361,25,1 +3,111,62,0,0,22.6,142,21,0 +9,123,70,44,94,33.1,374,40,0 +7,159,66,0,0,30.4,383,36,1 +11,135,0,0,0,52.3,578,40,1 +8,85,55,20,0,24.4,136,42,0 +5,158,84,41,210,39.4,395,29,1 +1,105,58,0,0,24.3,187,21,0 +3,107,62,13,48,22.9,678,23,1 +4,109,64,44,99,34.8,905,26,1 +4,148,60,27,318,30.9,150,29,1 +0,113,80,16,0,31.0,874,21,0 +1,138,82,0,0,40.1,236,28,0 +0,108,68,20,0,27.3,787,32,0 +2,99,70,16,44,20.4,235,27,0 +6,103,72,32,190,37.7,324,55,0 +5,111,72,28,0,23.9,407,27,0 +8,196,76,29,280,37.5,605,57,1 +5,162,104,0,0,37.7,151,52,1 +1,96,64,27,87,33.2,289,21,0 +7,184,84,33,0,35.5,355,41,1 +2,81,60,22,0,27.7,290,25,0 +0,147,85,54,0,42.8,375,24,0 +7,179,95,31,0,34.2,164,60,0 +0,140,65,26,130,42.6,431,24,1 +9,112,82,32,175,34.2,260,36,1 +12,151,70,40,271,41.8,742,38,1 +5,109,62,41,129,35.8,514,25,1 +6,125,68,30,120,30.0,464,32,0 +5,85,74,22,0,29.0,1224,32,1 +5,112,66,0,0,37.8,261,41,1 +0,177,60,29,478,34.6,1072,21,1 +2,158,90,0,0,31.6,805,66,1 +7,119,0,0,0,25.2,209,37,0 +7,142,60,33,190,28.8,687,61,0 +1,100,66,15,56,23.6,666,26,0 +1,87,78,27,32,34.6,101,22,0 +0,101,76,0,0,35.7,198,26,0 +3,162,52,38,0,37.2,652,24,1 +4,197,70,39,744,36.7,2329,31,0 +0,117,80,31,53,45.2,89,24,0 +4,142,86,0,0,44.0,645,22,1 +6,134,80,37,370,46.2,238,46,1 +1,79,80,25,37,25.4,583,22,0 +4,122,68,0,0,35.0,394,29,0 +3,74,68,28,45,29.7,293,23,0 +4,171,72,0,0,43.6,479,26,1 +7,181,84,21,192,35.9,586,51,1 +0,179,90,27,0,44.1,686,23,1 +9,164,84,21,0,30.8,831,32,1 +0,104,76,0,0,18.4,582,27,0 +1,91,64,24,0,29.2,192,21,0 +4,91,70,32,88,33.1,446,22,0 +3,139,54,0,0,25.6,402,22,1 +6,119,50,22,176,27.1,1318,33,1 +2,146,76,35,194,38.2,329,29,0 +9,184,85,15,0,30.0,1213,49,1 +10,122,68,0,0,31.2,258,41,0 +0,165,90,33,680,52.3,427,23,0 +9,124,70,33,402,35.4,282,34,0 +1,111,86,19,0,30.1,143,23,0 +9,106,52,0,0,31.2,380,42,0 +2,129,84,0,0,28.0,284,27,0 +2,90,80,14,55,24.4,249,24,0 +0,86,68,32,0,35.8,238,25,0 +12,92,62,7,258,27.6,926,44,1 +1,113,64,35,0,33.6,543,21,1 +3,111,56,39,0,30.1,557,30,0 +2,114,68,22,0,28.7,92,25,0 +1,193,50,16,375,25.9,655,24,0 +11,155,76,28,150,33.3,1353,51,1 +3,191,68,15,130,30.9,299,34,0 +3,141,0,0,0,30.0,761,27,1 +4,95,70,32,0,32.1,612,24,0 +3,142,80,15,0,32.4,200,63,0 +4,123,62,0,0,32.0,226,35,1 +5,96,74,18,67,33.6,997,43,0 +0,138,0,0,0,36.3,933,25,1 +2,128,64,42,0,40.0,1101,24,0 +0,102,52,0,0,25.1,78,21,0 +2,146,0,0,0,27.5,240,28,1 +10,101,86,37,0,45.6,1136,38,1 +2,108,62,32,56,25.2,128,21,0 +3,122,78,0,0,23.0,254,40,0 +1,71,78,50,45,33.2,422,21,0 +13,106,70,0,0,34.2,251,52,0 +2,100,70,52,57,40.5,677,25,0 +7,106,60,24,0,26.5,296,29,1 +0,104,64,23,116,27.8,454,23,0 +5,114,74,0,0,24.9,744,57,0 +2,108,62,10,278,25.3,881,22,0 +0,146,70,0,0,37.9,334,28,1 +10,129,76,28,122,35.9,280,39,0 +7,133,88,15,155,32.4,262,37,0 +7,161,86,0,0,30.4,165,47,1 +2,108,80,0,0,27.0,259,52,1 +7,136,74,26,135,26.0,647,51,0 +5,155,84,44,545,38.7,619,34,0 +1,119,86,39,220,45.6,808,29,1 +4,96,56,17,49,20.8,340,26,0 +5,108,72,43,75,36.1,263,33,0 +0,78,88,29,40,36.9,434,21,0 +0,107,62,30,74,36.6,757,25,1 +2,128,78,37,182,43.3,1224,31,1 +1,128,48,45,194,40.5,613,24,1 +0,161,50,0,0,21.9,254,65,0 +6,151,62,31,120,35.5,692,28,0 +2,146,70,38,360,28.0,337,29,1 +0,126,84,29,215,30.7,520,24,0 +14,100,78,25,184,36.6,412,46,1 +8,112,72,0,0,23.6,840,58,0 +0,167,0,0,0,32.3,839,30,1 +2,144,58,33,135,31.6,422,25,1 +5,77,82,41,42,35.8,156,35,0 +5,115,98,0,0,52.9,209,28,1 +3,150,76,0,0,21.0,207,37,0 +2,120,76,37,105,39.7,215,29,0 +10,161,68,23,132,25.5,326,47,1 +0,137,68,14,148,24.8,143,21,0 +0,128,68,19,180,30.5,1391,25,1 +2,124,68,28,205,32.9,875,30,1 +6,80,66,30,0,26.2,313,41,0 +0,106,70,37,148,39.4,605,22,0 +2,155,74,17,96,26.6,433,27,1 +3,113,50,10,85,29.5,626,25,0 +7,109,80,31,0,35.9,1127,43,1 +2,112,68,22,94,34.1,315,26,0 +3,99,80,11,64,19.3,284,30,0 +3,182,74,0,0,30.5,345,29,1 +3,115,66,39,140,38.1,150,28,0 +6,194,78,0,0,23.5,129,59,1 +4,129,60,12,231,27.5,527,31,0 +3,112,74,30,0,31.6,197,25,1 +0,124,70,20,0,27.4,254,36,1 +13,152,90,33,29,26.8,731,43,1 +2,112,75,32,0,35.7,148,21,0 +1,157,72,21,168,25.6,123,24,0 +1,122,64,32,156,35.1,692,30,1 +10,179,70,0,0,35.1,200,37,0 +2,102,86,36,120,45.5,127,23,1 +6,105,70,32,68,30.8,122,37,0 +8,118,72,19,0,23.1,1476,46,0 +2,87,58,16,52,32.7,166,25,0 +1,180,0,0,0,43.3,282,41,1 +12,106,80,0,0,23.6,137,44,0 +1,95,60,18,58,23.9,260,22,0 +0,165,76,43,255,47.9,259,26,0 +0,117,0,0,0,33.8,932,44,0 +5,115,76,0,0,31.2,343,44,1 +9,152,78,34,171,34.2,893,33,1 +7,178,84,0,0,39.9,331,41,1 +1,130,70,13,105,25.9,472,22,0 +1,95,74,21,73,25.9,673,36,0 +1,0,68,35,0,32.0,389,22,0 +5,122,86,0,0,34.7,290,33,0 +8,95,72,0,0,36.8,485,57,0 +8,126,88,36,108,38.5,349,49,0 +1,139,46,19,83,28.7,654,22,0 +3,116,0,0,0,23.5,187,23,0 +3,99,62,19,74,21.8,279,26,0 +5,0,80,32,0,41.0,346,37,1 +4,92,80,0,0,42.2,237,29,0 +4,137,84,0,0,31.2,252,30,0 +3,61,82,28,0,34.4,243,46,0 +1,90,62,12,43,27.2,580,24,0 +3,90,78,0,0,42.7,559,21,0 +9,165,88,0,0,30.4,302,49,1 +1,125,50,40,167,33.3,962,28,1 +13,129,0,30,0,39.9,569,44,1 +12,88,74,40,54,35.3,378,48,0 +1,196,76,36,249,36.5,875,29,1 +5,189,64,33,325,31.2,583,29,1 +5,158,70,0,0,29.8,207,63,0 +5,103,108,37,0,39.2,305,65,0 +4,146,78,0,0,38.5,520,67,1 +4,147,74,25,293,34.9,385,30,0 +5,99,54,28,83,34.0,499,30,0 +6,124,72,0,0,27.6,368,29,1 +0,101,64,17,0,21.0,252,21,0 +3,81,86,16,66,27.5,306,22,0 +1,133,102,28,140,32.8,234,45,1 +3,173,82,48,465,38.4,2137,25,1 +0,118,64,23,89,0.0,1731,21,0 +0,84,64,22,66,35.8,545,21,0 +2,105,58,40,94,34.9,225,25,0 +2,122,52,43,158,36.2,816,28,0 +12,140,82,43,325,39.2,528,58,1 +0,98,82,15,84,25.2,299,22,0 +1,87,60,37,75,37.2,509,22,0 +4,156,75,0,0,48.3,238,32,1 +0,93,100,39,72,43.4,1021,35,0 +1,107,72,30,82,30.8,821,24,0 +0,105,68,22,0,20.0,236,22,0 +1,109,60,8,182,25.4,947,21,0 +1,90,62,18,59,25.1,1268,25,0 +1,125,70,24,110,24.3,221,25,0 +1,119,54,13,50,22.3,205,24,0 +5,116,74,29,0,32.3,660,35,1 +8,105,100,36,0,43.3,239,45,1 +5,144,82,26,285,32.0,452,58,1 +3,100,68,23,81,31.6,949,28,0 +1,100,66,29,196,32.0,444,42,0 +5,166,76,0,0,45.7,340,27,1 +1,131,64,14,415,23.7,389,21,0 +4,116,72,12,87,22.1,463,37,0 +4,158,78,0,0,32.9,803,31,1 +2,127,58,24,275,27.7,1600,25,0 +3,96,56,34,115,24.7,944,39,0 +0,131,66,40,0,34.3,196,22,1 +3,82,70,0,0,21.1,389,25,0 +3,193,70,31,0,34.9,241,25,1 +4,95,64,0,0,32.0,161,31,1 +6,137,61,0,0,24.2,151,55,0 +5,136,84,41,88,35.0,286,35,1 +9,72,78,25,0,31.6,280,38,0 +5,168,64,0,0,32.9,135,41,1 +2,123,48,32,165,42.1,520,26,0 +4,115,72,0,0,28.9,376,46,1 +0,101,62,0,0,21.9,336,25,0 +8,197,74,0,0,25.9,1191,39,1 +1,172,68,49,579,42.4,702,28,1 +6,102,90,39,0,35.7,674,28,0 +1,112,72,30,176,34.4,528,25,0 +1,143,84,23,310,42.4,1076,22,0 +1,143,74,22,61,26.2,256,21,0 +0,138,60,35,167,34.6,534,21,1 +3,173,84,33,474,35.7,258,22,1 +1,97,68,21,0,27.2,1095,22,0 +4,144,82,32,0,38.5,554,37,1 +1,83,68,0,0,18.2,624,27,0 +3,129,64,29,115,26.4,219,28,1 +1,119,88,41,170,45.3,507,26,0 +2,94,68,18,76,26.0,561,21,0 +0,102,64,46,78,40.6,496,21,0 +2,115,64,22,0,30.8,421,21,0 +8,151,78,32,210,42.9,516,36,1 +4,184,78,39,277,37.0,264,31,1 +0,94,0,0,0,0.0,256,25,0 +1,181,64,30,180,34.1,328,38,1 +0,135,94,46,145,40.6,284,26,0 +1,95,82,25,180,35.0,233,43,1 +2,99,0,0,0,22.2,108,23,0 +3,89,74,16,85,30.4,551,38,0 +1,80,74,11,60,30.0,527,22,0 +2,139,75,0,0,25.6,167,29,0 +1,90,68,8,0,24.5,1138,36,0 +0,141,0,0,0,42.4,205,29,1 +12,140,85,33,0,37.4,244,41,0 +5,147,75,0,0,29.9,434,28,0 +1,97,70,15,0,18.2,147,21,0 +6,107,88,0,0,36.8,727,31,0 +0,189,104,25,0,34.3,435,41,1 +2,83,66,23,50,32.2,497,22,0 +4,117,64,27,120,33.2,230,24,0 +8,108,70,0,0,30.5,955,33,1 +4,117,62,12,0,29.7,380,30,1 +0,180,78,63,14,59.4,2420,25,1 +1,100,72,12,70,25.3,658,28,0 +0,95,80,45,92,36.5,330,26,0 +0,104,64,37,64,33.6,510,22,1 +0,120,74,18,63,30.5,285,26,0 +1,82,64,13,95,21.2,415,23,0 +2,134,70,0,0,28.9,542,23,1 +0,91,68,32,210,39.9,381,25,0 +2,119,0,0,0,19.6,832,72,0 +2,100,54,28,105,37.8,498,24,0 +14,175,62,30,0,33.6,212,38,1 +1,135,54,0,0,26.7,687,62,0 +5,86,68,28,71,30.2,364,24,0 +10,148,84,48,237,37.6,1001,51,1 +9,134,74,33,60,25.9,460,81,0 +9,120,72,22,56,20.8,733,48,0 +1,71,62,0,0,21.8,416,26,0 +8,74,70,40,49,35.3,705,39,0 +5,88,78,30,0,27.6,258,37,0 +10,115,98,0,0,24.0,1022,34,0 +0,124,56,13,105,21.8,452,21,0 +0,74,52,10,36,27.8,269,22,0 +0,97,64,36,100,36.8,600,25,0 +8,120,0,0,0,30.0,183,38,1 +6,154,78,41,140,46.1,571,27,0 +1,144,82,40,0,41.3,607,28,0 +0,137,70,38,0,33.2,170,22,0 +0,119,66,27,0,38.8,259,22,0 +7,136,90,0,0,29.9,210,50,0 +4,114,64,0,0,28.9,126,24,0 +0,137,84,27,0,27.3,231,59,0 +2,105,80,45,191,33.7,711,29,1 +7,114,76,17,110,23.8,466,31,0 +8,126,74,38,75,25.9,162,39,0 +4,132,86,31,0,28.0,419,63,0 +3,158,70,30,328,35.5,344,35,1 +0,123,88,37,0,35.2,197,29,0 +4,85,58,22,49,27.8,306,28,0 +0,84,82,31,125,38.2,233,23,0 +0,145,0,0,0,44.2,630,31,1 +0,135,68,42,250,42.3,365,24,1 +1,139,62,41,480,40.7,536,21,0 +0,173,78,32,265,46.5,1159,58,0 +4,99,72,17,0,25.6,294,28,0 +8,194,80,0,0,26.1,551,67,0 +2,83,65,28,66,36.8,629,24,0 +2,89,90,30,0,33.5,292,42,0 +4,99,68,38,0,32.8,145,33,0 +4,125,70,18,122,28.9,1144,45,1 +3,80,0,0,0,0.0,174,22,0 +6,166,74,0,0,26.6,304,66,0 +5,110,68,0,0,26.0,292,30,0 +2,81,72,15,76,30.1,547,25,0 +7,195,70,33,145,25.1,163,55,1 +6,154,74,32,193,29.3,839,39,0 +2,117,90,19,71,25.2,313,21,0 +3,84,72,32,0,37.2,267,28,0 +6,0,68,41,0,39.0,727,41,1 +7,94,64,25,79,33.3,738,41,0 +3,96,78,39,0,37.3,238,40,0 +10,75,82,0,0,33.3,263,38,0 +0,180,90,26,90,36.5,314,35,1 +1,130,60,23,170,28.6,692,21,0 +2,84,50,23,76,30.4,968,21,0 +8,120,78,0,0,25.0,409,64,0 +12,84,72,31,0,29.7,297,46,1 +0,139,62,17,210,22.1,207,21,0 +9,91,68,0,0,24.2,200,58,0 +2,91,62,0,0,27.3,525,22,0 +3,99,54,19,86,25.6,154,24,0 +3,163,70,18,105,31.6,268,28,1 +9,145,88,34,165,30.3,771,53,1 +7,125,86,0,0,37.6,304,51,0 +13,76,60,0,0,32.8,180,41,0 +6,129,90,7,326,19.6,582,60,0 +2,68,70,32,66,25.0,187,25,0 +3,124,80,33,130,33.2,305,26,0 +6,114,0,0,0,0.0,189,26,0 +9,130,70,0,0,34.2,652,45,1 +3,125,58,0,0,31.6,151,24,0 +3,87,60,18,0,21.8,444,21,0 +1,97,64,19,82,18.2,299,21,0 +3,116,74,15,105,26.3,107,24,0 +0,117,66,31,188,30.8,493,22,0 +0,111,65,0,0,24.6,660,31,0 +2,122,60,18,106,29.8,717,22,0 +0,107,76,0,0,45.3,686,24,0 +1,86,66,52,65,41.3,917,29,0 +6,91,0,0,0,29.8,501,31,0 +1,77,56,30,56,33.3,1251,24,0 +4,132,0,0,0,32.9,302,23,1 +0,105,90,0,0,29.6,197,46,0 +0,57,60,0,0,21.7,735,67,0 +0,127,80,37,210,36.3,804,23,0 +3,129,92,49,155,36.4,968,32,1 +8,100,74,40,215,39.4,661,43,1 +3,128,72,25,190,32.4,549,27,1 +10,90,85,32,0,34.9,825,56,1 +4,84,90,23,56,39.5,159,25,0 +1,88,78,29,76,32.0,365,29,0 +8,186,90,35,225,34.5,423,37,1 +5,187,76,27,207,43.6,1034,53,1 +4,131,68,21,166,33.1,160,28,0 +1,164,82,43,67,32.8,341,50,0 +4,189,110,31,0,28.5,680,37,0 +1,116,70,28,0,27.4,204,21,0 +3,84,68,30,106,31.9,591,25,0 +6,114,88,0,0,27.8,247,66,0 +1,88,62,24,44,29.9,422,23,0 +1,84,64,23,115,36.9,471,28,0 +7,124,70,33,215,25.5,161,37,0 +1,97,70,40,0,38.1,218,30,0 +8,110,76,0,0,27.8,237,58,0 +11,103,68,40,0,46.2,126,42,0 +11,85,74,0,0,30.1,300,35,0 +6,125,76,0,0,33.8,121,54,1 +0,198,66,32,274,41.3,502,28,1 +1,87,68,34,77,37.6,401,24,0 +6,99,60,19,54,26.9,497,32,0 +0,91,80,0,0,32.4,601,27,0 +2,95,54,14,88,26.1,748,22,0 +1,99,72,30,18,38.6,412,21,0 +6,92,62,32,126,32.0,85,46,0 +4,154,72,29,126,31.3,338,37,0 +0,121,66,30,165,34.3,203,33,1 +3,78,70,0,0,32.5,270,39,0 +2,130,96,0,0,22.6,268,21,0 +3,111,58,31,44,29.5,430,22,0 +2,98,60,17,120,34.7,198,22,0 +1,143,86,30,330,30.1,892,23,0 +1,119,44,47,63,35.5,280,25,0 +6,108,44,20,130,24.0,813,35,0 +2,118,80,0,0,42.9,693,21,1 +10,133,68,0,0,27.0,245,36,0 +2,197,70,99,0,34.7,575,62,1 +0,151,90,46,0,42.1,371,21,1 +6,109,60,27,0,25.0,206,27,0 +12,121,78,17,0,26.5,259,62,0 +8,100,76,0,0,38.7,190,42,0 +8,124,76,24,600,28.7,687,52,1 +1,93,56,11,0,22.5,417,22,0 +8,143,66,0,0,34.9,129,41,1 +6,103,66,0,0,24.3,249,29,0 +3,176,86,27,156,33.3,1154,52,1 +0,73,0,0,0,21.1,342,25,0 +11,111,84,40,0,46.8,925,45,1 +2,112,78,50,140,39.4,175,24,0 +3,132,80,0,0,34.4,402,44,1 +2,82,52,22,115,28.5,1699,25,0 +6,123,72,45,230,33.6,733,34,0 +0,188,82,14,185,32.0,682,22,1 +0,67,76,0,0,45.3,194,46,0 +1,89,24,19,25,27.8,559,21,0 +1,173,74,0,0,36.8,88,38,1 +1,109,38,18,120,23.1,407,26,0 +1,108,88,19,0,27.1,400,24,0 +6,96,0,0,0,23.7,190,28,0 +1,124,74,36,0,27.8,100,30,0 +7,150,78,29,126,35.2,692,54,1 +4,183,0,0,0,28.4,212,36,1 +1,124,60,32,0,35.8,514,21,0 +1,181,78,42,293,40.0,1258,22,1 +1,92,62,25,41,19.5,482,25,0 +0,152,82,39,272,41.5,270,27,0 +1,111,62,13,182,24.0,138,23,0 +3,106,54,21,158,30.9,292,24,0 +3,174,58,22,194,32.9,593,36,1 +7,168,88,42,321,38.2,787,40,1 +6,105,80,28,0,32.5,878,26,0 +11,138,74,26,144,36.1,557,50,1 +3,106,72,0,0,25.8,207,27,0 +6,117,96,0,0,28.7,157,30,0 +2,68,62,13,15,20.1,257,23,0 +9,112,82,24,0,28.2,1282,50,1 +0,119,0,0,0,32.4,141,24,1 +2,112,86,42,160,38.4,246,28,0 +2,92,76,20,0,24.2,1698,28,0 +6,183,94,0,0,40.8,1461,45,0 +0,94,70,27,115,43.5,347,21,0 +2,108,64,0,0,30.8,158,21,0 +4,90,88,47,54,37.7,362,29,0 +0,125,68,0,0,24.7,206,21,0 +0,132,78,0,0,32.4,393,21,0 +5,128,80,0,0,34.6,144,45,0 +4,94,65,22,0,24.7,148,21,0 +7,114,64,0,0,27.4,732,34,1 +0,102,78,40,90,34.5,238,24,0 +2,111,60,0,0,26.2,343,23,0 +1,128,82,17,183,27.5,115,22,0 +10,92,62,0,0,25.9,167,31,0 +13,104,72,0,0,31.2,465,38,1 +5,104,74,0,0,28.8,153,48,0 +2,94,76,18,66,31.6,649,23,0 +7,97,76,32,91,40.9,871,32,1 +1,100,74,12,46,19.5,149,28,0 +0,102,86,17,105,29.3,695,27,0 +4,128,70,0,0,34.3,303,24,0 +6,147,80,0,0,29.5,178,50,1 +4,90,0,0,0,28.0,610,31,0 +3,103,72,30,152,27.6,730,27,0 +2,157,74,35,440,39.4,134,30,0 +1,167,74,17,144,23.4,447,33,1 +0,179,50,36,159,37.8,455,22,1 +11,136,84,35,130,28.3,260,42,1 +0,107,60,25,0,26.4,133,23,0 +1,91,54,25,100,25.2,234,23,0 +1,117,60,23,106,33.8,466,27,0 +5,123,74,40,77,34.1,269,28,0 +2,120,54,0,0,26.8,455,27,0 +1,106,70,28,135,34.2,142,22,0 +2,155,52,27,540,38.7,240,25,1 +2,101,58,35,90,21.8,155,22,0 +1,120,80,48,200,38.9,1162,41,0 +11,127,106,0,0,39.0,190,51,0 +3,80,82,31,70,34.2,1292,27,1 +10,162,84,0,0,27.7,182,54,0 +1,199,76,43,0,42.9,1394,22,1 +8,167,106,46,231,37.6,165,43,1 +9,145,80,46,130,37.9,637,40,1 +6,115,60,39,0,33.7,245,40,1 +1,112,80,45,132,34.8,217,24,0 +4,145,82,18,0,32.5,235,70,1 +10,111,70,27,0,27.5,141,40,1 +6,98,58,33,190,34.0,430,43,0 +9,154,78,30,100,30.9,164,45,0 +6,165,68,26,168,33.6,631,49,0 +1,99,58,10,0,25.4,551,21,0 +10,68,106,23,49,35.5,285,47,0 +3,123,100,35,240,57.3,880,22,0 +8,91,82,0,0,35.6,587,68,0 +6,195,70,0,0,30.9,328,31,1 +9,156,86,0,0,24.8,230,53,1 +0,93,60,0,0,35.3,263,25,0 +3,121,52,0,0,36.0,127,25,1 +2,101,58,17,265,24.2,614,23,0 +2,56,56,28,45,24.2,332,22,0 +0,162,76,36,0,49.6,364,26,1 +0,95,64,39,105,44.6,366,22,0 +4,125,80,0,0,32.3,536,27,1 +5,136,82,0,0,0.0,640,69,0 +2,129,74,26,205,33.2,591,25,0 +3,130,64,0,0,23.1,314,22,0 +1,107,50,19,0,28.3,181,29,0 +1,140,74,26,180,24.1,828,23,0 +1,144,82,46,180,46.1,335,46,1 +8,107,80,0,0,24.6,856,34,0 +13,158,114,0,0,42.3,257,44,1 +2,121,70,32,95,39.1,886,23,0 +7,129,68,49,125,38.5,439,43,1 +2,90,60,0,0,23.5,191,25,0 +7,142,90,24,480,30.4,128,43,1 +3,169,74,19,125,29.9,268,31,1 +0,99,0,0,0,25.0,253,22,0 +4,127,88,11,155,34.5,598,28,0 +4,118,70,0,0,44.5,904,26,0 +2,122,76,27,200,35.9,483,26,0 +6,125,78,31,0,27.6,565,49,1 +1,168,88,29,0,35.0,905,52,1 +2,129,0,0,0,38.5,304,41,0 +4,110,76,20,100,28.4,118,27,0 +6,80,80,36,0,39.8,177,28,0 +10,115,0,0,0,0.0,261,30,1 +2,127,46,21,335,34.4,176,22,0 +9,164,78,0,0,32.8,148,45,1 +2,93,64,32,160,38.0,674,23,1 +3,158,64,13,387,31.2,295,24,0 +5,126,78,27,22,29.6,439,40,0 +10,129,62,36,0,41.2,441,38,1 +0,134,58,20,291,26.4,352,21,0 +3,102,74,0,0,29.5,121,32,0 +7,187,50,33,392,33.9,826,34,1 +3,173,78,39,185,33.8,970,31,1 +10,94,72,18,0,23.1,595,56,0 +1,108,60,46,178,35.5,415,24,0 +5,97,76,27,0,35.6,378,52,1 +4,83,86,19,0,29.3,317,34,0 +1,114,66,36,200,38.1,289,21,0 +1,149,68,29,127,29.3,349,42,1 +5,117,86,30,105,39.1,251,42,0 +1,111,94,0,0,32.8,265,45,0 +4,112,78,40,0,39.4,236,38,0 +1,116,78,29,180,36.1,496,25,0 +0,141,84,26,0,32.4,433,22,0 +2,175,88,0,0,22.9,326,22,0 +2,92,52,0,0,30.1,141,22,0 +3,130,78,23,79,28.4,323,34,1 +8,120,86,0,0,28.4,259,22,1 +2,174,88,37,120,44.5,646,24,1 +2,106,56,27,165,29.0,426,22,0 +2,105,75,0,0,23.3,560,53,0 +4,95,60,32,0,35.4,284,28,0 +0,126,86,27,120,27.4,515,21,0 +8,65,72,23,0,32.0,600,42,0 +2,99,60,17,160,36.6,453,21,0 +1,102,74,0,0,39.5,293,42,1 +11,120,80,37,150,42.3,785,48,1 +3,102,44,20,94,30.8,400,26,0 +1,109,58,18,116,28.5,219,22,0 +9,140,94,0,0,32.7,734,45,1 +13,153,88,37,140,40.6,1174,39,0 +12,100,84,33,105,30.0,488,46,0 +1,147,94,41,0,49.3,358,27,1 +1,81,74,41,57,46.3,1096,32,0 +3,187,70,22,200,36.4,408,36,1 +6,162,62,0,0,24.3,178,50,1 +4,136,70,0,0,31.2,1182,22,1 +1,121,78,39,74,39.0,261,28,0 +3,108,62,24,0,26.0,223,25,0 +0,181,88,44,510,43.3,222,26,1 +8,154,78,32,0,32.4,443,45,1 +1,128,88,39,110,36.5,1057,37,1 +7,137,90,41,0,32.0,391,39,0 +0,123,72,0,0,36.3,258,52,1 +1,106,76,0,0,37.5,197,26,0 +6,190,92,0,0,35.5,278,66,1 +2,88,58,26,16,28.4,766,22,0 +9,170,74,31,0,44.0,403,43,1 +9,89,62,0,0,22.5,142,33,0 +10,101,76,48,180,32.9,171,63,0 +2,122,70,27,0,36.8,340,27,0 +5,121,72,23,112,26.2,245,30,0 +1,126,60,0,0,30.1,349,47,1 +1,93,70,31,0,30.4,315,23,0 diff --git a/igel/examples/data/iris/eval-Iris.csv b/igel/examples/data/iris/eval-Iris.csv new file mode 100644 index 0000000..7f85d0d --- /dev/null +++ b/igel/examples/data/iris/eval-Iris.csv @@ -0,0 +1,38 @@ +SepalLengthCm,SepalWidthCm,PetalLengthCm,PetalWidthCm,Species +5.1,3.5,1.4,0.2,Iris-setosa +4.9,3.0,1.4,0.2,Iris-setosa +4.7,3.2,1.3,0.2,Iris-setosa +4.6,3.1,1.5,0.2,Iris-setosa +5.0,3.6,1.4,0.2,Iris-setosa +5.4,3.9,1.7,0.4,Iris-setosa +4.6,3.4,1.4,0.3,Iris-setosa +5.0,3.4,1.5,0.2,Iris-setosa +4.4,2.9,1.4,0.2,Iris-setosa +4.9,3.1,1.5,0.1,Iris-setosa +5.0,2.0,3.5,1.0,Iris-versicolor +5.9,3.0,4.2,1.5,Iris-versicolor +6.0,2.2,4.0,1.0,Iris-versicolor +6.1,2.9,4.7,1.4,Iris-versicolor +5.6,2.9,3.6,1.3,Iris-versicolor +6.7,3.1,4.4,1.4,Iris-versicolor +5.6,3.0,4.5,1.5,Iris-versicolor +5.8,2.7,4.1,1.0,Iris-versicolor +6.2,2.2,4.5,1.5,Iris-versicolor +5.6,2.5,3.9,1.1,Iris-versicolor +5.9,3.2,4.8,1.8,Iris-versicolor +6.3,3.3,6.0,2.5,Iris-virginica +5.8,2.7,5.1,1.9,Iris-virginica +7.1,3.0,5.9,2.1,Iris-virginica +6.3,2.9,5.6,1.8,Iris-virginica +6.5,3.0,5.8,2.2,Iris-virginica +7.6,3.0,6.6,2.1,Iris-virginica +4.9,2.5,4.5,1.7,Iris-virginica +7.3,2.9,6.3,1.8,Iris-virginica +6.7,2.5,5.8,1.8,Iris-virginica +7.2,3.6,6.1,2.5,Iris-virginica +6.5,3.2,5.1,2.0,Iris-virginica +6.4,2.7,5.3,1.9,Iris-virginica +6.8,3.0,5.5,2.1,Iris-virginica +5.7,2.5,5.0,2.0,Iris-virginica +5.8,2.8,5.1,2.4,Iris-virginica +6.4,3.2,5.3,2.3,Iris-virginica diff --git a/igel/examples/data/iris/test-Iris.csv b/igel/examples/data/iris/test-Iris.csv new file mode 100644 index 0000000..089bc47 --- /dev/null +++ b/igel/examples/data/iris/test-Iris.csv @@ -0,0 +1,38 @@ +SepalLengthCm,SepalWidthCm,PetalLengthCm,PetalWidthCm +5.1,3.5,1.4,0.2 +4.9,3.0,1.4,0.2 +4.7,3.2,1.3,0.2 +4.6,3.1,1.5,0.2 +5.0,3.6,1.4,0.2 +5.4,3.9,1.7,0.4 +4.6,3.4,1.4,0.3 +5.0,3.4,1.5,0.2 +4.4,2.9,1.4,0.2 +4.9,3.1,1.5,0.1 +5.0,2.0,3.5,1.0 +5.9,3.0,4.2,1.5 +6.0,2.2,4.0,1.0 +6.1,2.9,4.7,1.4 +5.6,2.9,3.6,1.3 +6.7,3.1,4.4,1.4 +5.6,3.0,4.5,1.5 +5.8,2.7,4.1,1.0 +6.2,2.2,4.5,1.5 +5.6,2.5,3.9,1.1 +5.9,3.2,4.8,1.8 +6.3,3.3,6.0,2.5 +5.8,2.7,5.1,1.9 +7.1,3.0,5.9,2.1 +6.3,2.9,5.6,1.8 +6.5,3.0,5.8,2.2 +7.6,3.0,6.6,2.1 +4.9,2.5,4.5,1.7 +7.3,2.9,6.3,1.8 +6.7,2.5,5.8,1.8 +7.2,3.6,6.1,2.5 +6.5,3.2,5.1,2.0 +6.4,2.7,5.3,1.9 +6.8,3.0,5.5,2.1 +5.7,2.5,5.0,2.0 +5.8,2.8,5.1,2.4 +6.4,3.2,5.3,2.3 diff --git a/igel/examples/data/iris/train-Iris.csv b/igel/examples/data/iris/train-Iris.csv new file mode 100644 index 0000000..70120eb --- /dev/null +++ b/igel/examples/data/iris/train-Iris.csv @@ -0,0 +1,151 @@ +SepalLengthCm,SepalWidthCm,PetalLengthCm,PetalWidthCm,Species +5.1,3.5,1.4,0.2,Iris-setosa +4.9,3.0,1.4,0.2,Iris-setosa +4.7,3.2,1.3,0.2,Iris-setosa +4.6,3.1,1.5,0.2,Iris-setosa +5.0,3.6,1.4,0.2,Iris-setosa +5.4,3.9,1.7,0.4,Iris-setosa +4.6,3.4,1.4,0.3,Iris-setosa +5.0,3.4,1.5,0.2,Iris-setosa +4.4,2.9,1.4,0.2,Iris-setosa +4.9,3.1,1.5,0.1,Iris-setosa +5.4,3.7,1.5,0.2,Iris-setosa +4.8,3.4,1.6,0.2,Iris-setosa +4.8,3.0,1.4,0.1,Iris-setosa +4.3,3.0,1.1,0.1,Iris-setosa +5.8,4.0,1.2,0.2,Iris-setosa +5.7,4.4,1.5,0.4,Iris-setosa +5.4,3.9,1.3,0.4,Iris-setosa +5.1,3.5,1.4,0.3,Iris-setosa +5.7,3.8,1.7,0.3,Iris-setosa +5.1,3.8,1.5,0.3,Iris-setosa +5.4,3.4,1.7,0.2,Iris-setosa +5.1,3.7,1.5,0.4,Iris-setosa +4.6,3.6,1.0,0.2,Iris-setosa +5.1,3.3,1.7,0.5,Iris-setosa +4.8,3.4,1.9,0.2,Iris-setosa +5.0,3.0,1.6,0.2,Iris-setosa +5.0,3.4,1.6,0.4,Iris-setosa +5.2,3.5,1.5,0.2,Iris-setosa +5.2,3.4,1.4,0.2,Iris-setosa +4.7,3.2,1.6,0.2,Iris-setosa +4.8,3.1,1.6,0.2,Iris-setosa +5.4,3.4,1.5,0.4,Iris-setosa +5.2,4.1,1.5,0.1,Iris-setosa +5.5,4.2,1.4,0.2,Iris-setosa +4.9,3.1,1.5,0.1,Iris-setosa +5.0,3.2,1.2,0.2,Iris-setosa +5.5,3.5,1.3,0.2,Iris-setosa +4.9,3.1,1.5,0.1,Iris-setosa +4.4,3.0,1.3,0.2,Iris-setosa +5.1,3.4,1.5,0.2,Iris-setosa +5.0,3.5,1.3,0.3,Iris-setosa +4.5,2.3,1.3,0.3,Iris-setosa +4.4,3.2,1.3,0.2,Iris-setosa +5.0,3.5,1.6,0.6,Iris-setosa +5.1,3.8,1.9,0.4,Iris-setosa +4.8,3.0,1.4,0.3,Iris-setosa +5.1,3.8,1.6,0.2,Iris-setosa +4.6,3.2,1.4,0.2,Iris-setosa +5.3,3.7,1.5,0.2,Iris-setosa +5.0,3.3,1.4,0.2,Iris-setosa +7.0,3.2,4.7,1.4,Iris-versicolor +6.4,3.2,4.5,1.5,Iris-versicolor +6.9,3.1,4.9,1.5,Iris-versicolor +5.5,2.3,4.0,1.3,Iris-versicolor +6.5,2.8,4.6,1.5,Iris-versicolor +5.7,2.8,4.5,1.3,Iris-versicolor +6.3,3.3,4.7,1.6,Iris-versicolor +4.9,2.4,3.3,1.0,Iris-versicolor +6.6,2.9,4.6,1.3,Iris-versicolor +5.2,2.7,3.9,1.4,Iris-versicolor +5.0,2.0,3.5,1.0,Iris-versicolor +5.9,3.0,4.2,1.5,Iris-versicolor +6.0,2.2,4.0,1.0,Iris-versicolor +6.1,2.9,4.7,1.4,Iris-versicolor +5.6,2.9,3.6,1.3,Iris-versicolor +6.7,3.1,4.4,1.4,Iris-versicolor +5.6,3.0,4.5,1.5,Iris-versicolor +5.8,2.7,4.1,1.0,Iris-versicolor +6.2,2.2,4.5,1.5,Iris-versicolor +5.6,2.5,3.9,1.1,Iris-versicolor +5.9,3.2,4.8,1.8,Iris-versicolor +6.1,2.8,4.0,1.3,Iris-versicolor +6.3,2.5,4.9,1.5,Iris-versicolor +6.1,2.8,4.7,1.2,Iris-versicolor +6.4,2.9,4.3,1.3,Iris-versicolor +6.6,3.0,4.4,1.4,Iris-versicolor +6.8,2.8,4.8,1.4,Iris-versicolor +6.7,3.0,5.0,1.7,Iris-versicolor +6.0,2.9,4.5,1.5,Iris-versicolor +5.7,2.6,3.5,1.0,Iris-versicolor +5.5,2.4,3.8,1.1,Iris-versicolor +5.5,2.4,3.7,1.0,Iris-versicolor +5.8,2.7,3.9,1.2,Iris-versicolor +6.0,2.7,5.1,1.6,Iris-versicolor +5.4,3.0,4.5,1.5,Iris-versicolor +6.0,3.4,4.5,1.6,Iris-versicolor +6.7,3.1,4.7,1.5,Iris-versicolor +6.3,2.3,4.4,1.3,Iris-versicolor +5.6,3.0,4.1,1.3,Iris-versicolor +5.5,2.5,4.0,1.3,Iris-versicolor +5.5,2.6,4.4,1.2,Iris-versicolor +6.1,3.0,4.6,1.4,Iris-versicolor +5.8,2.6,4.0,1.2,Iris-versicolor +5.0,2.3,3.3,1.0,Iris-versicolor +5.6,2.7,4.2,1.3,Iris-versicolor +5.7,3.0,4.2,1.2,Iris-versicolor +5.7,2.9,4.2,1.3,Iris-versicolor +6.2,2.9,4.3,1.3,Iris-versicolor +5.1,2.5,3.0,1.1,Iris-versicolor +5.7,2.8,4.1,1.3,Iris-versicolor +6.3,3.3,6.0,2.5,Iris-virginica +5.8,2.7,5.1,1.9,Iris-virginica +7.1,3.0,5.9,2.1,Iris-virginica +6.3,2.9,5.6,1.8,Iris-virginica +6.5,3.0,5.8,2.2,Iris-virginica +7.6,3.0,6.6,2.1,Iris-virginica +4.9,2.5,4.5,1.7,Iris-virginica +7.3,2.9,6.3,1.8,Iris-virginica +6.7,2.5,5.8,1.8,Iris-virginica +7.2,3.6,6.1,2.5,Iris-virginica +6.5,3.2,5.1,2.0,Iris-virginica +6.4,2.7,5.3,1.9,Iris-virginica +6.8,3.0,5.5,2.1,Iris-virginica +5.7,2.5,5.0,2.0,Iris-virginica +5.8,2.8,5.1,2.4,Iris-virginica +6.4,3.2,5.3,2.3,Iris-virginica +6.5,3.0,5.5,1.8,Iris-virginica +7.7,3.8,6.7,2.2,Iris-virginica +7.7,2.6,6.9,2.3,Iris-virginica +6.0,2.2,5.0,1.5,Iris-virginica +6.9,3.2,5.7,2.3,Iris-virginica +5.6,2.8,4.9,2.0,Iris-virginica +7.7,2.8,6.7,2.0,Iris-virginica +6.3,2.7,4.9,1.8,Iris-virginica +6.7,3.3,5.7,2.1,Iris-virginica +7.2,3.2,6.0,1.8,Iris-virginica +6.2,2.8,4.8,1.8,Iris-virginica +6.1,3.0,4.9,1.8,Iris-virginica +6.4,2.8,5.6,2.1,Iris-virginica +7.2,3.0,5.8,1.6,Iris-virginica +7.4,2.8,6.1,1.9,Iris-virginica +7.9,3.8,6.4,2.0,Iris-virginica +6.4,2.8,5.6,2.2,Iris-virginica +6.3,2.8,5.1,1.5,Iris-virginica +6.1,2.6,5.6,1.4,Iris-virginica +7.7,3.0,6.1,2.3,Iris-virginica +6.3,3.4,5.6,2.4,Iris-virginica +6.4,3.1,5.5,1.8,Iris-virginica +6.0,3.0,4.8,1.8,Iris-virginica +6.9,3.1,5.4,2.1,Iris-virginica +6.7,3.1,5.6,2.4,Iris-virginica +6.9,3.1,5.1,2.3,Iris-virginica +5.8,2.7,5.1,1.9,Iris-virginica +6.8,3.2,5.9,2.3,Iris-virginica +6.7,3.3,5.7,2.5,Iris-virginica +6.7,3.0,5.2,2.3,Iris-virginica +6.3,2.5,5.0,1.9,Iris-virginica +6.5,3.0,5.2,2.0,Iris-virginica +6.2,3.4,5.4,2.3,Iris-virginica +5.9,3.0,5.1,1.8,Iris-virginica diff --git a/igel/examples/hyperparams-search/fit.py b/igel/examples/hyperparams-search/fit.py new file mode 100644 index 0000000..f4acf46 --- /dev/null +++ b/igel/examples/hyperparams-search/fit.py @@ -0,0 +1,24 @@ +from igel import Igel + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel fit -dp path_to_dataset -yml path_to_yaml_file`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + +=============================================================================================================== + +This example fits a machine learning model on the indian-diabetes dataset + +- default model here is the neural network and the configuration are provided in neural-network.yaml file +- You can switch to random forest by providing the random-forest.yaml as the config file in the parameters + +""" + +mock_fit_params = {'data_path': '../data/indian-diabetes/train-indians-diabetes.csv', + 'yaml_path': './igel.yaml', + 'cmd': 'fit'} + +Igel(**mock_fit_params) + diff --git a/igel/examples/hyperparams-search/igel.yaml b/igel/examples/hyperparams-search/igel.yaml new file mode 100644 index 0000000..37d671d --- /dev/null +++ b/igel/examples/hyperparams-search/igel.yaml @@ -0,0 +1,32 @@ +# dataset operations +dataset: + type: csv + split: # split options + test_size: 0.2 # 0.2 means 20% for the test data, so 80% are automatically for training + shuffle: True # whether to shuffle the data before/while splitting + + preprocess: + scale: # scaling options + method: standard # standardization will scale values to have a 0 mean and 1 standard deviation | you can also try minmax + target: inputs # scale inputs. | other possible values: [outputs, all] # if you choose all then all values in the dataset will be scaled + + +# model definition +model: + type: classification + algorithm: RandomForest + hyperparameter_search: + method: random_search + parameter_grid: + max_depth: [6, 10] + n_estimators: [100, 300] + max_features: [auto, sqrt] + arguments: + cv: 5 + refit: true + return_train_score: false + verbose: 0 + +# target you want to predict +target: + - sick diff --git a/igel/examples/indian-diabetes-example/evaluate.py b/igel/examples/indian-diabetes-example/evaluate.py new file mode 100644 index 0000000..1a00199 --- /dev/null +++ b/igel/examples/indian-diabetes-example/evaluate.py @@ -0,0 +1,21 @@ +from igel import Igel + + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel evaluate -dp path_to_dataset`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + + +=============================================================================================================== + +This example uses the pre-fitted machine learning model to evaluate its performance + +""" + +mock_eval_params = {'data_path': '../data/indian-diabetes/eval-indians-diabetes.csv', + 'cmd': 'evaluate'} + +Igel(**mock_eval_params) diff --git a/igel/examples/indian-diabetes-example/fit.py b/igel/examples/indian-diabetes-example/fit.py new file mode 100644 index 0000000..c8c0ecb --- /dev/null +++ b/igel/examples/indian-diabetes-example/fit.py @@ -0,0 +1,26 @@ +from igel import Igel + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel fit -dp path_to_dataset -yml path_to_yaml_file`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + +=============================================================================================================== + +This example fits a machine learning model on the indian-diabetes dataset + +- default model here is the neural network and the configuration are provided in neural-network.yaml file +- You can switch to random forest by providing the random-forest.yaml as the config file in the parameters + +""" + +mock_fit_params = { + 'data_path': '../data/indian-diabetes/train-indians-diabetes.csv', + #'data_path': './data.json', + 'yaml_path': './neural-network.yaml', + 'cmd': 'fit'} + +Igel(**mock_fit_params) + diff --git a/igel/examples/indian-diabetes-example/neural-network.yaml b/igel/examples/indian-diabetes-example/neural-network.yaml new file mode 100644 index 0000000..5946c83 --- /dev/null +++ b/igel/examples/indian-diabetes-example/neural-network.yaml @@ -0,0 +1,13 @@ + + +dataset: + type: csv + +# model definition +model: + type: classification + algorithm: NeuralNetwork + +# target(s) you want to predict +target: + - sick diff --git a/igel/examples/indian-diabetes-example/predict.py b/igel/examples/indian-diabetes-example/predict.py new file mode 100644 index 0000000..68bef1d --- /dev/null +++ b/igel/examples/indian-diabetes-example/predict.py @@ -0,0 +1,21 @@ +from igel import Igel + + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel predict -dp path_to_dataset`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + + +=============================================================================================================== + +This example uses the pre-fitted machine learning model to generate predictions + +""" + +mock_pred_params = {'data_path': '../data/indian-diabetes/test-indians-diabetes.csv', + 'cmd': 'predict'} + +Igel(**mock_pred_params) diff --git a/igel/examples/indian-diabetes-example/random-forest.yaml b/igel/examples/indian-diabetes-example/random-forest.yaml new file mode 100644 index 0000000..2cb54ef --- /dev/null +++ b/igel/examples/indian-diabetes-example/random-forest.yaml @@ -0,0 +1,34 @@ + + +dataset: + type: csv + random_numbers: + generate_reproducible: true + seed: 42 + split: + test_size: 0.2 + shuffle: True + + preprocess: + missing_values: mean + encoding: + type: oneHotEncoding + scale: + method: standard + target: inputs + +model: + type: classification + algorithm: RandomForest + arguments: + n_estimators: 100 + max_depth: 30 + hyperparameter_search: + method: random_search + parameter_grid: + max_depth: [6, 10] + n_estimators: [100, 300] + max_features: [auto, sqrt] + +target: + - sick diff --git a/igel/examples/iris-example/evaluate.py b/igel/examples/iris-example/evaluate.py new file mode 100644 index 0000000..279867b --- /dev/null +++ b/igel/examples/iris-example/evaluate.py @@ -0,0 +1,16 @@ +from igel import Igel + + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel evaluate -dp path_to_dataset`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + +""" + +mock_eval_params = {'data_path': '../data/iris/eval-Iris.csv', + 'cmd': 'evaluate'} + +Igel(**mock_eval_params) diff --git a/igel/examples/iris-example/fit.py b/igel/examples/iris-example/fit.py new file mode 100644 index 0000000..14ad3d7 --- /dev/null +++ b/igel/examples/iris-example/fit.py @@ -0,0 +1,16 @@ +from igel import Igel + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel fit -dp path_to_dataset -yml path_to_yaml_file`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + +""" +mock_fit_params = {'data_path': '../data/iris/train-Iris.csv', + 'yaml_path': './iris.yaml', + 'cmd': 'fit'} + +Igel(**mock_fit_params) + diff --git a/igel/examples/iris-example/iris.yaml b/igel/examples/iris-example/iris.yaml new file mode 100644 index 0000000..53f9172 --- /dev/null +++ b/igel/examples/iris-example/iris.yaml @@ -0,0 +1,27 @@ +# dataset operations +dataset: + + split: # split options + test_size: 0.2 # 0.2 means 20% for the test data, so 80% are automatically for training + shuffle: True # whether to shuffle the data before/while splitting + stratify: default # If not None, data is split in a stratified fashion, using this as the class labels. + + preprocess: # preprocessing options + missing_values: mean # other possible values: [drop, median, most_frequent, constant] check the docs for more + encoding: + type: labelEncoding # other possible values: [labelEncoding] + column: Species + scale: # scaling options + method: standard # standardization will scale values to have a 0 mean and 1 standard deviation | you can also try minmax + target: inputs # scale inputs. | other possible values: [outputs, all] # if you choose all then all values in the dataset will be scaled + + +# model definition +model: + type: classification + algorithm: LogisticRegression + + +# target you want to predict +target: + - Species diff --git a/igel/examples/iris-example/predict.py b/igel/examples/iris-example/predict.py new file mode 100644 index 0000000..66f8e74 --- /dev/null +++ b/igel/examples/iris-example/predict.py @@ -0,0 +1,16 @@ +from igel import Igel + + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel predict -dp path_to_dataset`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + +""" + +mock_pred_params = {'data_path': '../data/iris/test-Iris.csv', + 'cmd': 'predict'} + +Igel(**mock_pred_params) diff --git a/igel/examples/multioutput-example/evaluate.py b/igel/examples/multioutput-example/evaluate.py new file mode 100644 index 0000000..6cbd98c --- /dev/null +++ b/igel/examples/multioutput-example/evaluate.py @@ -0,0 +1,16 @@ +from igel import Igel + + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel evaluate -dp path_to_dataset`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + +""" + +mock_eval_params = {'data_path': './linnerud.csv', + 'cmd': 'evaluate'} + +Igel(**mock_eval_params) diff --git a/igel/examples/multioutput-example/fit.py b/igel/examples/multioutput-example/fit.py new file mode 100644 index 0000000..dd72d3d --- /dev/null +++ b/igel/examples/multioutput-example/fit.py @@ -0,0 +1,16 @@ +from igel import Igel + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel fit -dp path_to_dataset -yml path_to_yaml_file`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + +""" +mock_fit_params = {'data_path': './linnerud.csv', + 'yaml_path': './multioutput.yaml', + 'cmd': 'fit'} + +Igel(**mock_fit_params) + diff --git a/igel/examples/multioutput-example/linnerud.csv b/igel/examples/multioutput-example/linnerud.csv new file mode 100644 index 0000000..fc3da13 --- /dev/null +++ b/igel/examples/multioutput-example/linnerud.csv @@ -0,0 +1,21 @@ +x1,x2,x3,y1,y2,y3 +5.0,162.0,60.0,191.0,36.0,50.0 +2.0,110.0,60.0,189.0,37.0,52.0 +12.0,101.0,101.0,193.0,38.0,58.0 +12.0,105.0,37.0,162.0,35.0,62.0 +13.0,155.0,58.0,189.0,35.0,46.0 +4.0,101.0,42.0,182.0,36.0,56.0 +8.0,101.0,38.0,211.0,38.0,56.0 +6.0,125.0,40.0,167.0,34.0,60.0 +15.0,200.0,40.0,176.0,31.0,74.0 +17.0,251.0,250.0,154.0,33.0,56.0 +17.0,120.0,38.0,169.0,34.0,50.0 +13.0,210.0,115.0,166.0,33.0,52.0 +14.0,215.0,105.0,154.0,34.0,64.0 +1.0,50.0,50.0,247.0,46.0,50.0 +6.0,70.0,31.0,193.0,36.0,46.0 +12.0,210.0,120.0,202.0,37.0,62.0 +4.0,60.0,25.0,176.0,37.0,54.0 +11.0,230.0,80.0,157.0,32.0,52.0 +15.0,225.0,73.0,156.0,33.0,54.0 +2.0,110.0,43.0,138.0,33.0,68.0 diff --git a/igel/examples/multioutput-example/multioutput.yaml b/igel/examples/multioutput-example/multioutput.yaml new file mode 100644 index 0000000..8058bac --- /dev/null +++ b/igel/examples/multioutput-example/multioutput.yaml @@ -0,0 +1,18 @@ +# dataset operations +dataset: + split: + test_size: 0.2 + shuffle: True + stratify: default +# scale: +# method: standard +# target: inputs + +model: + type: regression + algorithm: RandomForest + +target: + - y1 + - y2 + - y3 diff --git a/igel/examples/multioutput-example/predict.py b/igel/examples/multioutput-example/predict.py new file mode 100644 index 0000000..6a3468a --- /dev/null +++ b/igel/examples/multioutput-example/predict.py @@ -0,0 +1,16 @@ +from igel import Igel + + +""" +The goal of igel is to use ML without writing code. Therefore, the right and simplest way to use igel is from terminal. +You can run ` igel predict -dp path_to_dataset`. + +Alternatively, you can write code if you want. This example below demonstrates how to use igel if you want to write code. +However, I suggest you try and use the igel CLI. Type igel -h in your terminal to know more. + +""" + +mock_pred_params = {'data_path': './test-linnerud.csv', + 'cmd': 'predict'} + +Igel(**mock_pred_params) diff --git a/igel/examples/multioutput-example/test-linnerud.csv b/igel/examples/multioutput-example/test-linnerud.csv new file mode 100644 index 0000000..b80db4b --- /dev/null +++ b/igel/examples/multioutput-example/test-linnerud.csv @@ -0,0 +1,21 @@ +x1,x2,x3 +5,162,60 +2,110,60 +12,101,101 +12,105,37 +13,155,58 +4,101,42 +8,101,38 +6,125,40 +15,200,40 +17,251,250 +17,120,38 +13,210,115 +14,215,105 +1,50,50 +6,70,31 +12,210,120 +4,60,25 +11,230,80 +15,225,73 +2,110,43 diff --git a/igel/examples/python_client.py b/igel/examples/python_client.py new file mode 100644 index 0000000..ed28ed8 --- /dev/null +++ b/igel/examples/python_client.py @@ -0,0 +1,30 @@ +import json +import requests + +from igel.preprocessing import handle_missing_values, read_data_to_df + + +class IgelClient: + """ A mini client example that reads some data file and formats + it for the post call to predict + """ + def __init__(self, host, port, scheme="http", endpoint="predict", missing_values='mean'): + self.url = f"{scheme}://{host}:{port}/{endpoint}" + self.missing_values = missing_values + + def process_data(self, data_file): + """ Use the inbuilt handle_missing_values since missing values can't be sent in + valid JSON without raising an error + """ + data_df = handle_missing_values(read_data_to_df(data_file), strategy=self.missing_values) + + # make a dict of {header: column values}, JSONify, return + data_dict = data_df.to_dict(orient='list', into=dict) + jsonified = json.dumps(data_dict) + return jsonified + + def post(self, data_file): + output_json = self.process_data(data_file) + res = requests.post(self.url, data=output_json) + print(f"{res}: {res.text}") + return res diff --git a/igel/examples/regression_config.yaml b/igel/examples/regression_config.yaml new file mode 100644 index 0000000..a5363e5 --- /dev/null +++ b/igel/examples/regression_config.yaml @@ -0,0 +1,8 @@ +input: data/regression_data.csv +target: price +model: + type: regression + algorithm: LinearRegression + params: + fit_intercept: true + normalize: false diff --git a/igel/heathcare.py b/igel/heathcare.py new file mode 100644 index 0000000..36b9fd5 --- /dev/null +++ b/igel/heathcare.py @@ -0,0 +1,19 @@ +""" +Healthcare ML Utilities + +- HIPAA-compliant data handling (placeholder) +- Medical image analysis tools (placeholder) +- Clinical decision support systems (placeholder) +""" + +def is_hipaa_compliant(data): + # Placeholder: Add real HIPAA checks here + return True + +def analyze_medical_image(image_path): + # Placeholder: Add real image analysis here + return {"diagnosis": "normal", "confidence": 0.99} + +def clinical_decision_support(patient_data): + # Placeholder: Add real decision support logic here + return {"recommendation": "No action needed"} diff --git a/igel/igel/SECURE_MPC.PY b/igel/igel/SECURE_MPC.PY new file mode 100644 index 0000000..2d3dd22 --- /dev/null +++ b/igel/igel/SECURE_MPC.PY @@ -0,0 +1,19 @@ +""" +Secure Multi-party Computation (MPC) Utilities + +- Secure aggregation for federated learning (placeholder) +- Homomorphic encryption support (placeholder) +- Secure model sharing protocols (placeholder) +""" + +def secure_aggregate(parties_data): + # Placeholder: Add real secure aggregation logic here + return sum(parties_data) / len(parties_data) + +def homomorphic_encrypt(data, public_key): + # Placeholder: Add real homomorphic encryption logic here + return f"encrypted({data})" + +def secure_model_share(model, recipients): + # Placeholder: Add real secure sharing logic here + return {"shared_with": recipients, "status": "success"} diff --git a/igel/igel/__init__.py b/igel/igel/__init__.py new file mode 100644 index 0000000..9e8dbbf Binary files /dev/null and b/igel/igel/__init__.py differ diff --git a/igel/igel/__main__.py b/igel/igel/__main__.py new file mode 100644 index 0000000..76b5593 --- /dev/null +++ b/igel/igel/__main__.py @@ -0,0 +1,700 @@ +"""Console script for igel.""" +import logging +import os +import subprocess +from pathlib import Path +import json + +import click +import igel +import pandas as pd +from igel import Igel, metrics_dict +from igel.constants import Constants +from igel.servers import fastapi_server +from igel.utils import print_models_overview, show_model_info, tableize +from igel.model_registry import models_dict + +logger = logging.getLogger(__name__) +CONTEXT_SETTINGS = dict(help_option_names=["-h", "--help"]) + + +@click.group() +@click.option('--verbose', is_flag=True, help='Enable verbose output for debugging.') +def cli(verbose): + """ + The igel command line interface + """ + if verbose: + logging.basicConfig(level=logging.DEBUG) + logger.setLevel(logging.DEBUG) + click.echo('Verbose mode is on.') + else: + logging.basicConfig(level=logging.WARNING) + logger.setLevel(logging.WARNING) + + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option( + "--model_type", + "-type", + type=click.Choice(Constants.supported_model_types, case_sensitive=False), + help="type of the problem you want to solve", +) +@click.option( + "--model_name", + "-name", + help="algorithm you want to use", +) +@click.option( + "--target", + "-tg", + help="target you want to predict (this is usually the name of column you want to predict)", +) +def init(model_type: str, model_name: str, target: str) -> None: + """ + Initialize a new igel project. + This command can be run interactively or by providing command line arguments. + + Example: + igel init + igel init --model_type=classification --model_name=RandomForest --target=label + """ + if not model_type: + model_type = click.prompt( + "Please choose a model type", + type=click.Choice(Constants.supported_model_types, case_sensitive=False) + ) + + algorithms = models_dict.get(model_type, {}) + available_algorithms = list(algorithms.keys()) + + if not model_name: + model_name = click.prompt( + "Please choose an algorithm", + type=click.Choice(available_algorithms, case_sensitive=False) + ) + + if not target: + target = click.prompt("Please enter the target column(s) you want to predict (comma-separated)") + + Igel.create_init_mock_file( + model_type=model_type, model_name=model_name, target=target + ) + + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option( + "--data_path", "-dp", required=True, help="Path to your training dataset" +) +@click.option( + "--yaml_path", + "-yml", + required=True, + help="Path to your igel configuration file (yaml or json file)", +) +def fit(data_path: str, yaml_path: str) -> None: + """ + Fit/train a machine learning model. + + Example: + igel fit --data_path=data/train.csv --yaml_path=config.yaml + """ + Igel(cmd="fit", data_path=data_path, yaml_path=yaml_path) + + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option( + "--data_path", "-dp", required=True, help="Path to your evaluation dataset" +) +def evaluate(data_path: str) -> None: + """ + Evaluate the performance of an existing machine learning model + """ + Igel(cmd="evaluate", data_path=data_path) + + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option("--data_path", "-dp", required=True, help="Path to your dataset") +def predict(data_path: str) -> None: + """ + Use an existing machine learning model to generate predictions + """ + Igel(cmd="predict", data_path=data_path) + + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option( + "--data_paths", + "-DP", + required=True, + help="Path to your datasets as string separated by space", +) +@click.option( + "--yaml_path", + "-yml", + required=True, + help="Path to your igel configuration file (yaml or json file)", +) +def experiment(data_paths: str, yaml_path: str) -> None: + """ + train, evaluate and use pre-trained model for predictions in one command + """ + train_data_path, eval_data_path, pred_data_path = data_paths.strip().split( + " " + ) + Igel(cmd="fit", data_path=train_data_path, yaml_path=yaml_path) + Igel(cmd="evaluate", data_path=eval_data_path) + Igel(cmd="predict", data_path=pred_data_path) + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option("--model_path", "-dp", required=True, help="Path to your sklearn model") +def export(model_path: str) -> None: + """ + Export an existing machine learning model to ONNX + """ + Igel(cmd="export", model_path=model_path) + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option( + "--model_results_dir", + "-res_dir", + required=True, + help="Path to your model results directory", +) +@click.option( + "--host", "-h", default="localhost", show_default=True, help="server host" +) +@click.option( + "--port", "-p", default="8080", show_default=True, help="server host" +) +def serve(model_results_dir: str, host: str, port: int): + """ + expose a REST endpoint in order to use the trained machine learning model + """ + try: + os.environ[Constants.model_results_path] = model_results_dir + uvicorn_params = {"host": host, "port": int(port)} + fastapi_server.run(**uvicorn_params) + + except Exception as ex: + logger.exception(ex) + + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option( + "--model_type", + "-type", + type=click.Choice(Constants.supported_model_types, case_sensitive=False), + help="type of the model you want to inspect", +) +@click.option("--model_name", "-name", help="algorithm you want to use") +def models(model_type: str, model_name: str) -> None: + """ + show an overview of all models supported by igel + """ + if not model_type or not model_name: + print_models_overview() + else: + show_model_info(model_type=model_type, model_name=model_name) + + +@cli.command(context_settings=CONTEXT_SETTINGS) +def metrics(): + """ + show an overview of all metrics supported by igel + """ + print(f"\nIgel's supported metrics overview: \n") + reg_metrics = [func.__name__ for func in metrics_dict.get("regression")] + clf_metrics = [func.__name__ for func in metrics_dict.get("classification")] + + df_metrics = ( + pd.DataFrame.from_dict( + {"regression": reg_metrics, "classification": clf_metrics}, + orient="index", + ) + .transpose() + .fillna("----") + ) + + df_metrics = tableize(df_metrics) + print(df_metrics) + + +@cli.command(context_settings=CONTEXT_SETTINGS) +def gui(): + """ + Launch the igel gui application. + PS: you need to have nodejs on your machine + """ + igel_ui_path = Path(os.getcwd()) / "igel-ui" + if not Path.exists(igel_ui_path): + subprocess.check_call( + ["git"] + ["clone", "https://github.com/nidhaloff/igel-ui.git"] + ) + logger.info(f"igel UI cloned successfully") + + os.chdir(igel_ui_path) + logger.info(f"switching to -> {igel_ui_path}") + logger.info(f"current dir: {os.getcwd()}") + logger.info(f"make sure you have nodejs installed!!") + + subprocess.Popen(["node", "npm", "install", "open"], shell=True) + subprocess.Popen(["node", "npm", "install electron", "open"], shell=True) + logger.info("installing dependencies ...") + logger.info(f"dependencies installed successfully") + logger.info(f"node version:") + subprocess.check_call("node -v", shell=True) + logger.info(f"npm version:") + subprocess.check_call("npm -v", shell=True) + subprocess.check_call("npm i electron", shell=True) + logger.info("running igel UI...") + subprocess.check_call("npm start", shell=True) + + +@cli.command(context_settings=CONTEXT_SETTINGS) +def help(): + """get help about how to use igel""" + with click.Context(cli) as ctx: + click.echo(cli.get_help(ctx)) + + +@cli.command(context_settings=CONTEXT_SETTINGS) +def version(): + """ + Show the current igel version. + """ + click.echo(f"igel version: {igel.__version__}") + + +@cli.command(context_settings=CONTEXT_SETTINGS) +def info(): + """get info & metadata about igel""" + print( + f""" + package name: igel + version: {igel.__version__} + author: Nidhal Baccouri + maintainer: Nidhal Baccouri + contact: nidhalbacc@gmail.com + license: MIT + description: use machine learning without writing code + dependencies: pandas, sklearn, pyyaml + requires python: >= 3.6 + First release: 27.08.2020 + official repo: https://github.com/nidhaloff/igel + written in: 100% python + status: stable + operating system: independent + """ + ) + + +@cli.group() +def registry(): + """ + Manage model registry operations + """ + pass + +@registry.command(context_settings=CONTEXT_SETTINGS) +@click.option( + "--model_name", + "-name", + help="Filter models by name", +) +def list_models(model_name: str) -> None: + """ + List all registered models or filter by name + """ + models = Igel.list_models(model_name) + if not models: + click.echo("No models found in registry.") + return + + # Create a table of models + table_data = [] + for model in models: + table_data.append({ + "ID": model["model_id"], + "Name": model["model_name"], + "Version": model["version"], + "Type": model["model_type"], + "Algorithm": model["model_algorithm"], + "Created": model["timestamp"] + }) + + df = pd.DataFrame(table_data) + click.echo(tableize(df)) + +@registry.command(context_settings=CONTEXT_SETTINGS) +@click.option( + "--model_id", + "-id", + required=True, + help="Model ID to get information about", +) +def info(model_id: str) -> None: + """ + Get detailed information about a specific model + """ + model_info = Igel.get_model_info(model_id) + if not model_info: + click.echo(f"No model found with ID: {model_id}") + return + + click.echo("\nModel Information:") + for key, value in model_info.items(): + if isinstance(value, dict): + click.echo(f"\n{key}:") + for k, v in value.items(): + click.echo(f" {k}: {v}") + else: + click.echo(f"{key}: {value}") + +@registry.command(context_settings=CONTEXT_SETTINGS) +@click.option( + "--model_id", + "-id", + required=True, + help="Model ID to delete", +) +def delete(model_id: str) -> None: + """ + Delete a model from the registry + """ + if Igel.delete_model(model_id): + click.echo(f"Successfully deleted model: {model_id}") + else: + click.echo(f"Failed to delete model: {model_id}") + +@registry.command(context_settings=CONTEXT_SETTINGS) +@click.option( + "--model_id", + "-id", + required=True, + help="Model ID to update", +) +@click.option( + "--metadata", + "-m", + required=True, + help="JSON string containing metadata to update", +) +def update(model_id: str, metadata: str) -> None: + """ + Update metadata for a registered model + """ + try: + metadata_dict = json.loads(metadata) + if Igel.update_model_metadata(model_id, metadata_dict): + click.echo(f"Successfully updated metadata for model: {model_id}") + else: + click.echo(f"Failed to update metadata for model: {model_id}") + except json.JSONDecodeError: + click.echo("Error: Invalid JSON format for metadata") + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option( + "--model_a_path", + "-ma", + required=True, + help="Path to first model for comparison", +) +@click.option( + "--model_b_path", + "-mb", + required=True, + help="Path to second model for comparison", +) +@click.option( + "--test_data", + "-td", + required=True, + help="Path to test dataset for comparison", +) +@click.option( + "--problem_type", + "-pt", + type=click.Choice(["classification", "regression"]), + default="classification", + help="Type of problem (classification or regression)", +) +def compare_models(model_a_path: str, model_b_path: str, test_data: str, problem_type: str) -> None: + """ + Compare two trained models using A/B testing framework. + + This command loads two trained models and compares their performance using + statistical tests to determine if there are significant differences. + """ + try: + from igel.ab_testing import ModelComparison + import joblib + import pandas as pd + + # Load models + model_a = joblib.load(model_a_path) + model_b = joblib.load(model_b_path) + + # Load test data + test_df = pd.read_csv(test_data) + X_test = test_df.drop(columns=['target']).values # Assuming 'target' is the label column + y_test = test_df['target'].values + + # Initialize comparison + comparison = ModelComparison(model_a, model_b, test_type=problem_type) + + # Run comparison + results = comparison.compare_predictions(X_test, y_test) + + # Generate and print report + report = comparison.generate_report(results) + print("\n" + report + "\n") + + except Exception as e: + logger.exception(f"Error during model comparison: {e}") + raise click.ClickException(str(e)) + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option('--ref_data', required=True, help='Path to reference (e.g., training) data CSV') +@click.option('--new_data', required=True, help='Path to new (e.g., production) data CSV') +@click.option('--categorical', default='', help='Comma-separated list of categorical feature names') +def detect_drift(ref_data, new_data, categorical): + """ + Detect data drift between two datasets (reference and new). + Uses KS test for numerical and Chi-Squared for categorical features. + """ + import pandas as pd + from igel.drift_detection import detect_drift + ref_df = pd.read_csv(ref_data) + new_df = pd.read_csv(new_data) + categorical_features = [c.strip() for c in categorical.split(',') if c.strip()] if categorical else None + report = detect_drift(ref_df, new_df, categorical_features) + print(report.to_string(index=False)) + +@cli.command(context_settings=CONTEXT_SETTINGS) +def gpu_info(): + """ + Show GPU availability and utilization (PyTorch, TensorFlow, GPUtil). + """ + from igel.gpu_utils import detect_gpu, report_gpu_utilization + print(detect_gpu()) + report_gpu_utilization() + +@cli.command() +@click.option('--results_dir', default='results', help='Directory with model result files') +def leaderboard(results_dir): + """Generate a leaderboard comparing all trained models.""" + from igel.leaderboard import generate_leaderboard + generate_leaderboard(results_dir) + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option('--action', required=True, + type=click.Choice(['optimize', 'allocate', 'simulate']), + help='Action to perform: optimize trajectory, allocate resources, or simulate mission') +@click.option('--config_path', required=True, help='Path to mission configuration JSON file') +@click.option('--output_path', default='mission_results.json', help='Output file path') +def space_mission(action, config_path, output_path): + """ + Perform space mission planning operations. + """ + import json + from igel.space_mission import optimize_trajectory, allocate_resources, simulate_mission + + # Load mission configuration + with open(config_path, 'r') as f: + config = json.load(f) + + # Perform requested action + if action == 'optimize': + start_point = config.get('start_point', [0, 0, 0]) + end_point = config.get('end_point', [1000, 0, 0]) + constraints = config.get('constraints', {}) + results = optimize_trajectory(start_point, end_point, constraints) + + elif action == 'allocate': + mission_goals = config.get('mission_goals', []) + available_resources = config.get('available_resources', {}) + results = allocate_resources(mission_goals, available_resources) + + else: # simulate + mission_plan = config.get('mission_plan', {}) + environment_params = config.get('environment_params', {}) + results = simulate_mission(mission_plan, environment_params) + + # Save results + with open(output_path, 'w') as f: + json.dump(results, f, indent=2) + + print(f"Space mission {action} completed. Results saved to {output_path}") + print(f"Results: {results}") + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option('--data_path', '-dp', required=True, help='Path to your dataset') +@click.option('--yaml_path', '-yml', required=True, help='Path to your igel configuration file') +@click.option('--n_way', default=2, help='Number of classes per task (default: 2)') +@click.option('--k_shot', default=5, help='Number of examples per class for support set (default: 5)') +@click.option('--n_query', default=5, help='Number of examples per class for query set (default: 5)') +@click.option('--num_tasks', default=10, help='Number of tasks to create (default: 10)') +def few_shot_learn(data_path, yaml_path, n_way, k_shot, n_query, num_tasks): + """ + Train a few-shot learning model using MAML or Prototypical Networks. + + Example: + igel few-shot-learn --data_path=data/train.csv --yaml_path=config.yaml --n_way=3 --k_shot=5 + """ + try: + from igel.few_shot_learning import create_few_shot_dataset, evaluate_few_shot_model + import pandas as pd + + # Load data + df = pd.read_csv(data_path) + target_col = 'target' # This should be configurable from yaml + X = df.drop(columns=[target_col]).values + y = df[target_col].values + + # Create few-shot tasks + tasks = create_few_shot_dataset(X, y, n_way=n_way, k_shot=k_shot, n_query=n_query) + + # Load configuration + file_ext = yaml_path.split(".")[-1] + if file_ext == "yaml": + from igel.utils import read_yaml + config = read_yaml(yaml_path) + else: + from igel.utils import read_json + config = read_json(yaml_path) + + # Get model configuration + model_props = config.get("model", {}) + model_type = model_props.get("type") + model_algorithm = model_props.get("algorithm") + + if model_type != "few_shot_learning": + raise ValueError("Model type must be 'few_shot_learning' for few-shot learning") + + # Create model + if model_algorithm == "MAML": + from igel.few_shot_learning import MAMLClassifier + model = MAMLClassifier( + inner_lr=model_props.get("arguments", {}).get("inner_lr", 0.01), + outer_lr=model_props.get("arguments", {}).get("outer_lr", 0.001), + num_tasks=model_props.get("arguments", {}).get("num_tasks", 10), + shots_per_task=model_props.get("arguments", {}).get("shots_per_task", 5), + inner_steps=model_props.get("arguments", {}).get("inner_steps", 5), + meta_epochs=model_props.get("arguments", {}).get("meta_epochs", 100) + ) + elif model_algorithm == "PrototypicalNetwork": + from igel.few_shot_learning import PrototypicalNetwork + model = PrototypicalNetwork( + embedding_dim=model_props.get("arguments", {}).get("embedding_dim", 64), + num_tasks=model_props.get("arguments", {}).get("num_tasks", 10), + shots_per_task=model_props.get("arguments", {}).get("shots_per_task", 5), + meta_epochs=model_props.get("arguments", {}).get("meta_epochs", 100) + ) + else: + raise ValueError(f"Unsupported few-shot learning algorithm: {model_algorithm}") + + # Train model + print(f"Training {model_algorithm} model...") + model.fit(X, y) + + # Evaluate on few-shot tasks + print("Evaluating model on few-shot tasks...") + results = evaluate_few_shot_model(model, tasks) + + print(f"\nFew-shot learning results:") + print(f"Mean accuracy: {results['mean_accuracy']:.4f}") + print(f"Std accuracy: {results['std_accuracy']:.4f}") + + # Save model + import joblib + joblib.dump(model, "few_shot_model.joblib") + print("Model saved as 'few_shot_model.joblib'") + + except Exception as e: + logger.exception(f"Error during few-shot learning: {e}") + raise click.ClickException(str(e)) + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option('--source_data', required=True, help='Path to source domain data') +@click.option('--target_data', required=True, help='Path to target domain data') +@click.option('--method', default='fine_tune', + type=click.Choice(['fine_tune', 'domain_adversarial', 'maml']), + help='Domain adaptation method') +@click.option('--output_model', default='adapted_model.joblib', help='Output model path') +def domain_adapt(source_data, target_data, method, output_model): + """ + Perform domain adaptation from source to target domain. + + Example: + igel domain-adapt --source_data=source.csv --target_data=target.csv --method=fine_tune + """ + try: + from igel.few_shot_learning import DomainAdaptation + import pandas as pd + import joblib + + # Load data + source_df = pd.read_csv(source_data) + target_df = pd.read_csv(target_data) + + # Assume target column is 'target' + source_X = source_df.drop(columns=['target']).values + source_y = source_df['target'].values + target_X = target_df.drop(columns=['target']).values + target_y = target_df['target'].values if 'target' in target_df.columns else None + + # Create a base model (you could load a pre-trained model here) + from sklearn.ensemble import RandomForestClassifier + base_model = RandomForestClassifier(random_state=42) + + # Perform domain adaptation + adapter = DomainAdaptation(base_model) + adapted_model = adapter.adapt_model(source_X, source_y, target_X, target_y, method) + + # Save adapted model + joblib.dump(adapted_model, output_model) + print(f"Domain adapted model saved to {output_model}") + + except Exception as e: + logger.exception(f"Error during domain adaptation: {e}") + raise click.ClickException(str(e)) + +@cli.command(context_settings=CONTEXT_SETTINGS) +@click.option('--source_model', required=True, help='Path to pre-trained source model') +@click.option('--target_data', required=True, help='Path to target data') +@click.option('--method', default='feature_extraction', + type=click.Choice(['feature_extraction', 'fine_tuning']), + help='Transfer learning method') +@click.option('--output_model', default='transfer_model.joblib', help='Output model path') +def transfer_learn(source_model, target_data, method, output_model): + """ + Perform transfer learning using a pre-trained model. + + Example: + igel transfer-learn --source_model=pretrained.joblib --target_data=new_data.csv + """ + try: + from igel.few_shot_learning import TransferLearning + import pandas as pd + import joblib + + # Load pre-trained model + source_model = joblib.load(source_model) + + # Load target data + target_df = pd.read_csv(target_data) + target_X = target_df.drop(columns=['target']).values + target_y = target_df['target'].values + + # Perform transfer learning + transfer = TransferLearning(source_model) + transfer_model = transfer.create_transfer_model(source_model, target_X, target_y, method) + + # Save transfer model + joblib.dump(transfer_model, output_model) + print(f"Transfer learning model saved to {output_model}") + + except Exception as e: + logger.exception(f"Error during transfer learning: {e}") + raise click.ClickException(str(e)) diff --git a/igel/igel/ab_testing.py b/igel/igel/ab_testing.py new file mode 100644 index 0000000..aaed798 --- /dev/null +++ b/igel/igel/ab_testing.py @@ -0,0 +1,142 @@ +""" +Module for A/B testing of machine learning models. +Provides functionality to compare model versions and assess statistical significance. +""" + +import numpy as np +from scipy import stats +import pandas as pd +from typing import List, Dict, Tuple, Union +from sklearn.metrics import accuracy_score, mean_squared_error, r2_score +import logging + +logger = logging.getLogger(__name__) + +class ModelComparison: + """Class to handle A/B testing between two model versions.""" + + def __init__(self, model_a, model_b, test_type: str = "classification"): + """ + Initialize model comparison. + + Args: + model_a: First model to compare + model_b: Second model to compare + test_type: Type of models being compared ("classification" or "regression") + """ + self.model_a = model_a + self.model_b = model_b + self.test_type = test_type + + def compare_predictions(self, X_test: np.ndarray, y_true: np.ndarray) -> Dict: + """ + Compare predictions from both models. + + Args: + X_test: Test features + y_true: True labels/values + + Returns: + Dict containing comparison metrics + """ + # Get predictions from both models + y_pred_a = self.model_a.predict(X_test) + y_pred_b = self.model_b.predict(X_test) + + # Calculate metrics based on problem type + if self.test_type == "classification": + metrics_a = { + "accuracy": accuracy_score(y_true, y_pred_a), + "predictions": y_pred_a + } + metrics_b = { + "accuracy": accuracy_score(y_true, y_pred_b), + "predictions": y_pred_b + } + + # Perform McNemar's test for statistical significance + contingency_table = self._create_contingency_table(y_true, y_pred_a, y_pred_b) + stat, p_value = stats.mcnemar(contingency_table, correction=True) + + else: # regression + metrics_a = { + "mse": mean_squared_error(y_true, y_pred_a), + "r2": r2_score(y_true, y_pred_a), + "predictions": y_pred_a + } + metrics_b = { + "mse": mean_squared_error(y_true, y_pred_b), + "r2": r2_score(y_true, y_pred_b), + "predictions": y_pred_b + } + + # Perform Wilcoxon signed-rank test + stat, p_value = stats.wilcoxon( + (y_true - y_pred_a)**2, + (y_true - y_pred_b)**2 + ) + + return { + "model_a_metrics": metrics_a, + "model_b_metrics": metrics_b, + "statistical_test": { + "statistic": stat, + "p_value": p_value, + "significant_difference": p_value < 0.05 + } + } + + def _create_contingency_table( + self, y_true: np.ndarray, y_pred_a: np.ndarray, y_pred_b: np.ndarray + ) -> np.ndarray: + """ + Create contingency table for McNemar's test. + + Returns: + 2x2 contingency table as numpy array + """ + # Both correct + n11 = sum((y_pred_a == y_true) & (y_pred_b == y_true)) + # A correct, B wrong + n12 = sum((y_pred_a == y_true) & (y_pred_b != y_true)) + # A wrong, B correct + n21 = sum((y_pred_a != y_true) & (y_pred_b == y_true)) + # Both wrong + n22 = sum((y_pred_a != y_true) & (y_pred_b != y_true)) + + return np.array([[n11, n12], [n21, n22]]) + + def generate_report(self, comparison_results: Dict) -> str: + """ + Generate a formatted report of the comparison results. + + Args: + comparison_results: Results from compare_predictions + + Returns: + Formatted string containing the comparison report + """ + report = [] + report.append("Model A/B Testing Results") + report.append("=" * 25) + + # Add metrics based on problem type + if self.test_type == "classification": + report.append(f"\nModel A Accuracy: {comparison_results['model_a_metrics']['accuracy']:.4f}") + report.append(f"Model B Accuracy: {comparison_results['model_b_metrics']['accuracy']:.4f}") + else: + report.append(f"\nModel A MSE: {comparison_results['model_a_metrics']['mse']:.4f}") + report.append(f"Model A R²: {comparison_results['model_a_metrics']['r2']:.4f}") + report.append(f"Model B MSE: {comparison_results['model_b_metrics']['mse']:.4f}") + report.append(f"Model B R²: {comparison_results['model_b_metrics']['r2']:.4f}") + + # Add statistical test results + report.append("\nStatistical Test Results") + report.append("-" * 22) + report.append(f"Test Statistic: {comparison_results['statistical_test']['statistic']:.4f}") + report.append(f"P-value: {comparison_results['statistical_test']['p_value']:.4f}") + report.append( + f"Significant Difference: {'Yes' if comparison_results['statistical_test']['significant_difference'] else 'No'}" + ) + + return "\n".join(report) \ No newline at end of file diff --git a/igel/igel/anomaly_detection.py b/igel/igel/anomaly_detection.py new file mode 100644 index 0000000..bc08559 --- /dev/null +++ b/igel/igel/anomaly_detection.py @@ -0,0 +1,59 @@ +""" +Time Series Anomaly Detection Utilities + +- Isolation Forest for anomaly detection +- Statistical methods for outlier detection +- Time series specific anomaly detection +""" + +import numpy as np +from sklearn.ensemble import IsolationForest +from sklearn.preprocessing import StandardScaler + +def detect_anomalies_isolation_forest(data, contamination=0.1): + """ + Detect anomalies using Isolation Forest. + + Args: + data: Time series data (1D array) + contamination: Expected proportion of anomalies + + Returns: + Dictionary with anomaly scores and predictions + """ + # Reshape data for sklearn + X = data.reshape(-1, 1) + + # Fit isolation forest + iso_forest = IsolationForest(contamination=contamination, random_state=42) + predictions = iso_forest.fit_predict(X) + scores = iso_forest.score_samples(X) + + return { + "predictions": predictions, # -1 for anomalies, 1 for normal + "scores": scores, + "anomaly_indices": np.where(predictions == -1)[0] + } + +def detect_statistical_anomalies(data, threshold=3): + """ + Detect anomalies using statistical methods (Z-score). + + Args: + data: Time series data + threshold: Number of standard deviations for anomaly detection + + Returns: + Dictionary with anomaly information + """ + mean = np.mean(data) + std = np.std(data) + z_scores = np.abs((data - mean) / std) + + anomalies = z_scores > threshold + + return { + "anomaly_indices": np.where(anomalies)[0], + "z_scores": z_scores, + "threshold": threshold + } \ No newline at end of file diff --git a/igel/igel/auto/__init__.py b/igel/igel/auto/__init__.py new file mode 100644 index 0000000..0a97e9e --- /dev/null +++ b/igel/igel/auto/__init__.py @@ -0,0 +1,3 @@ +from .cnn import IgelCNN + +__all__ = ["IgelCNN"] diff --git a/igel/igel/auto/cnn.py b/igel/igel/auto/cnn.py new file mode 100644 index 0000000..02d20c5 --- /dev/null +++ b/igel/igel/auto/cnn.py @@ -0,0 +1,141 @@ +import json +import logging +import os + +import autokeras as ak +from igel.auto.defaults import Defaults +from igel.auto.models import Models +from igel.constants import Constants +from igel.utils import read_json, read_yaml +from tensorflow.keras.models import load_model + +logger = logging.getLogger(__name__) + + +class IgelCNN: + defaults = Defaults() + model = None + dataset_props = {} + model_props = {} + model_args = {} + training_args = {} + results_path = Constants.results_dir + + def __init__(self, **cli_args): + self.cmd: str = cli_args.get("cmd") + self.data_path: str = cli_args.get("data_path") + self.config_path: str = cli_args.get("yaml_path") + self.task = cli_args.get("task") + logger.info(f"Executing command: {self.cmd}") + logger.info(f"Reading data from: {self.data_path}") + logger.info(f"Reading yaml configs from: {self.config_path}") + + if self.cmd == "train": + if not self.config_path: + self.model_type = self.task + else: + self.file_ext: str = self.config_path.split(".")[1] + + if self.file_ext not in ["yaml", "yml", "json"]: + raise Exception( + "Configuration file can be a yaml or a json file!" + ) + + self.configs: dict = ( + read_json(self.config_path) + if self.file_ext == "json" + else read_yaml(self.config_path) + ) + + self.dataset_props: dict = self.configs.get( + "dataset", self.defaults.dataset_props + ) + self.model_props: dict = self.configs.get( + "model", self.defaults.model_props + ) + self.training_args: dict = self.configs.get( + "training", self.defaults.training_args + ) + self.model_args = self.model_props.get("arguments") + self.model_type = self.task or self.model_props.get("type") + + else: + self.model_path = cli_args.get( + "model_path", self.defaults.model_path + ) + logger.info(f"path of the pre-fitted model => {self.model_path}") + self.prediction_file = cli_args.get( + "prediction_file", self.defaults.prediction_file + ) + # set description.json if provided: + self.description_file = cli_args.get( + "description_file", self.defaults.description_file + ) + # load description file to read stored training parameters + with open(self.description_file) as f: + dic = json.load(f) + self.model_type: str = dic.get("task") # type of the model + self.dataset_props: dict = dic.get( + "dataset_props" + ) # dataset props entered while fitting + getattr(self, self.cmd)() + + def _create_model(self, *args, **kwargs): + model_cls = Models.get(self.model_type) + model = ( + model_cls() if not self.model_args else model_cls(**self.model_args) + ) + return model + + def save_desc_file(self): + desc = { + "task": self.model_type, + "model": self.model.__class__.__name__, + "dataset_props": self.dataset_props or None, + "model_props": self.model_props or None, + } + try: + logger.info(f"saving fit description to {self.description_file}") + with open(self.description_file, "w", encoding="utf-8") as f: + json.dump(desc, f, ensure_ascii=False, indent=4) + except Exception as e: + logger.exception( + f"Error while storing the fit description file: {e}" + ) + + def save_model(self): + exp_model = self.model.export_model() + logger.info(f"model type: {type(exp_model)}") + try: + exp_model.save("model", save_format="tf") + return True + except Exception: + exp_model.save(f"model.h5") + + def train(self): + train_data = ak.image_dataset_from_directory(self.data_path) + self.model = self._create_model() + logger.info(f"executing a {self.model.__class__.__name__} algorithm...") + logger.info(f"Training started...") + self.model.fit(train_data, **self.training_args) + logger.info("finished training!") + self.save_desc_file() + saved = self.save_model() + if saved: + logger.info(f"model saved successfully") + + def load_model(self): + logger.info("loading model...") + loaded_model = load_model("model", custom_objects=ak.CUSTOM_OBJECTS) + logger.info("model loaded successfully") + return loaded_model + + def evaluate(self): + trained_model = self.load_model() + test_data = ak.image_dataset_from_directory(self.data_path) + trained_model.evaluate(test_data) + + def predict(self): + trained_model = self.load_model() + pred_data = ak.image_dataset_from_directory(self.data_path) + trained_model.predict(pred_data) diff --git a/igel/igel/auto/defaults.py b/igel/igel/auto/defaults.py new file mode 100644 index 0000000..341a4cd --- /dev/null +++ b/igel/igel/auto/defaults.py @@ -0,0 +1,22 @@ +from igel.configs import configs + + +class Defaults: + dataset_props = {} + model_props = {} + training_args = {} + available_commands = ("fit", "evaluate", "predict", "experiment","export") + supported_types = ("regression", "classification", "clustering") + results_path = configs.get("results_path") # path to the results folder + model_path = configs.get( + "default_model_path" + ) # path to the pre-fitted model + description_file = configs.get( + "description_file" + ) # path to the description.json file + evaluation_file = configs.get( + "evaluation_file" + ) # path to the evaluation.json file + prediction_file = configs.get( + "prediction_file" + ) # path to the predictions.csv diff --git a/igel/igel/auto/example.py b/igel/igel/auto/example.py new file mode 100644 index 0000000..0c5ef07 --- /dev/null +++ b/igel/igel/auto/example.py @@ -0,0 +1,9 @@ +import autokeras as ak +import tensorflow as tf +from tensorflow.keras.datasets import mnist +from tensorflow.keras.models import load_model + +(x_train, y_train), (x_test, y_test) = mnist.load_data() + +cls = ak.ImageClassifier() +cls.fit(x_train, y_train) diff --git a/igel/igel/auto/models.py b/igel/igel/auto/models.py new file mode 100644 index 0000000..ab0595c --- /dev/null +++ b/igel/igel/auto/models.py @@ -0,0 +1,39 @@ +import autokeras as ak + + +class Models: + models_map = { + "ImageClassification": { + "class": ak.ImageClassifier, + "link": "https://autokeras.com/image_classifier/", + }, + "ImageRegression": { + "class": ak.ImageRegressor, + "link": "https://autokeras.com/image_regressor/", + }, + "TextClassification": { + "class": ak.TextClassifier, + "link": "https://autokeras.com/text_classifier/", + }, + "TextRegression": { + "class": ak.TextRegressor, + "link": "https://autokeras.com/text_regressor/", + }, + "StructuredDataClassification": { + "class": ak.StructuredDataClassifier, + "link": "https://autokeras.com/structured_data_classifier/", + }, + "StructuredDataRegression": { + "class": ak.StructuredDataRegressor, + "link": "https://autokeras.com/structured_data_regressor/", + }, + } + + @classmethod + def get(cls, model_type: str, *args, **kwargs): + if model_type not in cls.models_map.keys(): + raise Exception( + f"{model_type} is not supported! " + f"Choose one of the following supported tasks: {cls.models_map.keys()}" + ) + return cls.models_map[model_type]["class"] diff --git a/igel/igel/configs.py b/igel/igel/configs.py new file mode 100644 index 0000000..e4b3eb8 --- /dev/null +++ b/igel/igel/configs.py @@ -0,0 +1,44 @@ +import os +from pathlib import Path + +from igel.constants import Constants + +res_path = Path(os.getcwd()) / Constants.stats_dir +init_file_path = Path(os.getcwd()) / Constants.init_file +temp_post_req_data_path = Path(os.getcwd()) / Constants.post_req_data_file + +configs = { + "stats_dir": Constants.stats_dir, + "model_file": Constants.model_file, + "results_path": res_path, + "default_model_path": res_path / Constants.model_file, + "default_onnx_model_path": res_path / Constants.onnx_model_file, + "description_file": res_path / Constants.description_file, + "evaluation_file": res_path / Constants.evaluation_file, + "prediction_file": res_path / Constants.prediction_file, + "init_file_path": init_file_path, + "dataset_props": { + "type": "csv", + "split": {"test_size": 0.1, "shuffle": True}, + "preprocess": { + "missing_values": "mean", + "scale": {"method": "standard", "target": "inputs"}, + }, + }, + "model_props": {"type": "classification", "algorithm": "NeuralNetwork"}, + "available_dataset_props": { + "type": "csv", + "separator": ",", + "split": {"test_size": None, "shuffle": False, "stratify": None}, + "preprocess": { + "missing_values": "mean", + "encoding": None, + "scale": None, + }, + }, + "available_model_props": { + "type": "regression", + "algorithm": "linear regression", + "arguments": "default", + }, +} diff --git a/igel/igel/constants.py b/igel/igel/constants.py new file mode 100644 index 0000000..ec2a3b2 --- /dev/null +++ b/igel/igel/constants.py @@ -0,0 +1,12 @@ +class Constants: + model_results_path = "IGEL_MODEL_RESULTS_PATH" + model_file = "model.joblib" + onnx_model_file = "model.onnx" + description_file = "description.json" + prediction_file = "predictions.csv" + stats_dir = "model_results" + results_dir = "model_results" + init_file = "igel.yaml" + post_req_data_file = "post_req_data.csv" + evaluation_file = "evaluation.json" + supported_model_types = ["regression", "classification", "clustering", "few_shot_learning"] diff --git a/igel/igel/custom_model.py b/igel/igel/custom_model.py new file mode 100644 index 0000000..5b57441 --- /dev/null +++ b/igel/igel/custom_model.py @@ -0,0 +1,13 @@ +class CustomModelBase: + """ + Base class for custom model architectures in igel. + Users can inherit from this class to define their own models. + Required methods: + - fit(X, y): Train the model on data X and labels y. + - predict(X): Predict using the trained model on data X. + """ + def fit(self, X, y): + raise NotImplementedError("fit method must be implemented by the custom model.") + + def predict(self, X): + raise NotImplementedError("predict method must be implemented by the custom model.") \ No newline at end of file diff --git a/igel/igel/data.py b/igel/igel/data.py new file mode 100644 index 0000000..71e8e18 --- /dev/null +++ b/igel/igel/data.py @@ -0,0 +1,562 @@ +from sklearn.linear_model import (LinearRegression, + LogisticRegression, + Ridge, + RANSACRegressor, + RidgeClassifier, + RidgeClassifierCV, + RidgeCV, + BayesianRidge, + SGDRegressor, + GammaRegressor, + LogisticRegressionCV, + TheilSenRegressor, + PoissonRegressor, + TweedieRegressor, + ARDRegression, + SGDClassifier, + HuberRegressor, + Lasso, + LassoCV, + LassoLars, + LassoLarsCV, + PassiveAggressiveClassifier, + ElasticNet, + ElasticNetCV, + Perceptron) + +from sklearn.ensemble import (RandomForestClassifier, + RandomForestRegressor, + ExtraTreesRegressor, + ExtraTreesClassifier, + AdaBoostClassifier, + AdaBoostRegressor, + BaggingClassifier, + BaggingRegressor, + GradientBoostingClassifier, + GradientBoostingRegressor, + StackingClassifier, + StackingRegressor, + VotingClassifier, + VotingRegressor) + +from sklearn.naive_bayes import (BernoulliNB, + CategoricalNB, + ComplementNB, + GaussianNB, + MultinomialNB) + +from sklearn.cluster import (KMeans, + AffinityPropagation, + AgglomerativeClustering, + Birch, + DBSCAN, + FeatureAgglomeration, + MiniBatchKMeans, + MeanShift, + OPTICS, + SpectralBiclustering, + SpectralClustering, + SpectralCoclustering) + +from sklearn.calibration import CalibratedClassifierCV +from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor, ExtraTreeClassifier, ExtraTreeRegressor +from sklearn.svm import SVC, SVR, LinearSVC, LinearSVR, NuSVC, NuSVR +from sklearn.neighbors import KNeighborsClassifier, KNeighborsRegressor +from sklearn.neural_network import MLPRegressor, MLPClassifier, BernoulliRBM +from sklearn.metrics import (mean_squared_error, + mean_absolute_error, + mean_squared_log_error, + median_absolute_error, + accuracy_score, + f1_score, + r2_score, + precision_score, + recall_score) +from sklearn.utils.multiclass import type_of_target + +from igel.extras.kmedoids import KMedoids +from igel.extras.kmedians import KMedians +from igel.few_shot_learning import MAMLClassifier, PrototypicalNetwork + +import pandas as pd + +import logging + +logging.basicConfig(format='%(levelname)s - %(message)s', level=logging.INFO) +logger = logging.getLogger(__name__) + +models_dict = { + + "regression": { + + "LinearRegression": { + "class": LinearRegression, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html", + + }, + + "SGDRegressor": { + "class": SGDRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDRegressor.html?" + "highlight=sgd#sklearn.linear_model.SGDRegressor", + + }, + + "Lasso": { + "class": Lasso, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html?" + "highlight=lasso#sklearn.linear_model.Lasso", + "cv_class": LassoCV + }, + + "LassoLars": { + "class": LassoLars, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LassoLars.html?" + "highlight=lasso#sklearn.linear_model.LassoLars", + "cv_class": LassoLarsCV + }, + + "BayesianRegression": { + "class": BayesianRidge, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.BayesianRidge.html?" + "highlight=ridge#sklearn.linear_model.BayesianRidge" + }, + + "HuberRegression": { + "class": HuberRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.HuberRegressor.html?" + "highlight=huber#sklearn.linear_model.HuberRegressor" + }, + + "Ridge": { + "class": Ridge, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html" + "#sklearn.linear_model.Ridge", + "cv_class": RidgeCV + }, + + "PoissonRegression": { + "class": PoissonRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.PoissonRegressor.html?" + "highlight=poisson#sklearn.linear_model.PoissonRegressor" + }, + + "ARDRegression": { + "class": ARDRegression, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.ARDRegression.html?" + "highlight=ard#sklearn.linear_model.ARDRegression" + }, + + "TweedieRegression": { + "class": TweedieRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.TweedieRegressor.html?" + "highlight=tweedie#sklearn.linear_model.TweedieRegressor" + }, + + "TheilSenRegression": { + "class": TheilSenRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.TheilSenRegressor.html?" + "highlight=theilsenregressor#sklearn.linear_model.TheilSenRegressor" + }, + + "GammaRegression": { + "class": GammaRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.GammaRegressor.html?" + "highlight=gamma%20regressor#sklearn.linear_model.GammaRegressor" + }, + + "RANSACRegression": { + "class": RANSACRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.RANSACRegressor.html" + }, + + "DecisionTree": { + "class": DecisionTreeRegressor, + "link": "https://scikit-learn.org/stable/modules/generatedsklearn.tree.DecisionTreeRegressor.html?" + "highlight=decision%20tree%20regressor#sklearn.tree.DecisionTreeRegressor" + }, + + "ExtraTree": { + "class": ExtraTreeRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.tree.ExtraTreeRegressor.html" + "#sklearn.tree.ExtraTreeRegressor" + }, + + "RandomForest": { + "class": RandomForestRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html?" + "highlight=random%20forest#sklearn.ensemble.RandomForestRegressor"}, + + "ExtraTrees": { + "class": ExtraTreesRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.ExtraTreesRegressor.html?" + "highlight=extra%20trees#sklearn.ensemble.ExtraTreesRegressor"}, + + "SVM": { + "class": SVR, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVR.html?" + "highlight=svr#sklearn.svm.SVR"}, + + "LinearSVM": { + "class": LinearSVR, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVR.html#sklearn.svm.LinearSVR" + }, + + "NuSVM": { + "class": NuSVR, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.svm.NuSVR.html#sklearn.svm.NuSVR" + }, + + "NearestNeighbor": { + "class": KNeighborsRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsRegressor.html?" + "highlight=neighbor#sklearn.neighbors.KNeighborsRegressor"}, + + "NeuralNetwork": { + "class": MLPRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html?" + "highlight=mlp#sklearn.neural_network.MLPRegressor" + }, + + "ElasticNet": { + "class": ElasticNet, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.ElasticNet.html?" + "highlight=elasticnet#sklearn.linear_model.ElasticNet", + "cv_class": ElasticNetCV + }, + + "BernoulliRBM": { + "class": BernoulliRBM, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.BernoulliRBM.html#" + "sklearn.neural_network.BernoulliRBM" + }, + + "BoltzmannMachine": { + "class": BernoulliRBM, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.BernoulliRBM.html#" + "sklearn.neural_network.BernoulliRBM" + }, + + "Adaboost": { + "class": AdaBoostRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostRegressor.html#" + "sklearn.ensemble.AdaBoostRegressor" + }, + + "Bagging": { + "class": BaggingRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingRegressor.html" + "#sklearn.ensemble.BaggingRegressor" + }, + + "GradientBoosting": { + "class": GradientBoostingRegressor, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html" + "#sklearn.ensemble.GradientBoostingRegressor" + } + }, + + "classification": { + + "LogisticRegression": { + + "class": LogisticRegression, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html?" + "highlight=regression#sklearn.linear_model.LogisticRegression", + "cv_class": LogisticRegressionCV + }, + + "SGDClassifier": { + + "class": SGDClassifier, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html?" + "highlight=sgd#sklearn.linear_model.SGDClassifier", + }, + + "Ridge": { + "class": RidgeClassifier, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.RidgeClassifier.html?" + "highlight=ridgeclassifier#sklearn.linear_model.RidgeClassifier", + "cv_class": RidgeClassifierCV + }, + + "DecisionTree": { + "class": DecisionTreeClassifier, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html?" + "highlight=decision%20tree#sklearn.tree.DecisionTreeClassifier"}, + + "ExtraTree": { + "class": ExtraTreeClassifier, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.tree.ExtraTreeClassifier.html" + "#sklearn.tree.ExtraTreeClassifier" + }, + + "RandomForest": { + "class": RandomForestClassifier, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html?" + "highlight=random%20forest#sklearn.ensemble.RandomForestClassifier"}, + + "ExtraTrees": { + "class": ExtraTreesClassifier, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.ExtraTreesClassifier.html?" + "highlight=extra%20trees#sklearn.ensemble.ExtraTreesClassifier"}, + + "SVM": { + "class": SVC, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html?" + "highlight=svc#sklearn.svm.SVC"}, + + "LinearSVM": { + "class": LinearSVC, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html#sklearn.svm.LinearSVC" + }, + + "NuSVM": { + "class": NuSVC, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.svm.NuSVC.html#sklearn.svm.NuSVC" + }, + + + "NearestNeighbor": { + "class": KNeighborsClassifier, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html?" + "highlight=neighbor#sklearn.neighbors.KNeighborsClassifier"}, + + "NeuralNetwork": { + "class": MLPClassifier, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html?" + "highlight=mlp#sklearn.neural_network.MLPClassifier" + }, + + "PassiveAgressiveClassifier": { + "class": PassiveAggressiveClassifier, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.PassiveAggressiveClassifier.html?" + "highlight=passiveaggressiveclassifier#sklearn.linear_model.PassiveAggressiveClassifier" + }, + + "Perceptron": { + "class": Perceptron, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Perceptron.html#" + "sklearn.linear_model.Perceptron" + }, + + "BernoulliRBM": { + "class": BernoulliRBM, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.BernoulliRBM.html#" + "sklearn.neural_network.BernoulliRBM" + }, + + "BoltzmannMachine": { + "class": BernoulliRBM, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.BernoulliRBM.html#" + "sklearn.neural_network.BernoulliRBM" + }, + + "CalibratedClassifier": { + "class": CalibratedClassifierCV, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.calibration.CalibratedClassifierCV.html#" + "sklearn.calibration.CalibratedClassifierCV" + }, + + "Adaboost": { + "class": AdaBoostClassifier, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostClassifier.html#" + "sklearn.ensemble.AdaBoostClassifier" + }, + + "Bagging": { + "class": BaggingClassifier, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingClassifier.html" + "#sklearn.ensemble.BaggingClassifier" + }, + + "GradientBoosting": { + "class": GradientBoostingClassifier, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html" + "#sklearn.ensemble.GradientBoostingClassifier" + }, + + "BernoulliNaiveBayes": { + "class": BernoulliNB, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.BernoulliNB.html" + "#sklearn.naive_bayes.BernoulliNB" + }, + + "CategoricalNaiveBayes": { + "class": CategoricalNB, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.CategoricalNB.html" + "#sklearn.naive_bayes.CategoricalNB" + }, + + "ComplementNaiveBayes": { + "class": ComplementNB, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.ComplementNB.html" + "#sklearn.naive_bayes.ComplementNB" + }, + + "GaussianNaiveBayes": { + "class": GaussianNB, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.GaussianNB.html" + "#sklearn.naive_bayes.GaussianNB" + }, + + "MultinomialNaiveBayes": { + "class": MultinomialNB, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html" + "#sklearn.naive_bayes.MultinomialNB" + } + + + }, + + "clustering": { + "KMeans": { + "class": KMeans, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html" + "#sklearn.cluster.KMeans" + }, + + "KMedoids": { + "class": KMedoids, + "link": "https://scikit-learn-extra.readthedocs.io/en/stable/generated/sklearn_extra.cluster.KMedoids.html" + "#igel.extras.KMedoids" + }, + + "KMedians": { + "class": KMedians, + "link": "Local Implementation" + "#igel.extras.KMedians" + }, + + "AffinityPropagation": { + "class": AffinityPropagation, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.cluster.AffinityPropagation.html" + "#sklearn.cluster.AffinityPropagation" + }, + + "Birch": { + "class": Birch, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.cluster.Birch.html#sklearn.cluster.Birch" + }, + + "AgglomerativeClustering": { + "class": AgglomerativeClustering, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.cluster.AgglomerativeClustering.html#" + "sklearn.cluster.AgglomerativeClustering" + }, + + "FeatureAgglomeration": { + "class": FeatureAgglomeration, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.cluster.FeatureAgglomeration.html#" + "sklearn.cluster.FeatureAgglomeration" + }, + "DBSCAN": { + "class": DBSCAN, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html" + "#sklearn.cluster.DBSCAN" + }, + "MiniBatchKMeans": { + "class": MiniBatchKMeans, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.cluster.MiniBatchKMeans.html" + "#sklearn.cluster.MiniBatchKMeans" + }, + + "SpectralBiclustering": { + "class": SpectralBiclustering, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.cluster.SpectralBiclustering.html" + "#sklearn.cluster.SpectralBiclustering" + }, + + "SpectralCoclustering": { + "class": SpectralCoclustering, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.cluster.SpectralCoclustering.html" + "#sklearn.cluster.SpectralCoclustering" + }, + + "SpectralClustering": { + "class": SpectralClustering, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.cluster.SpectralClustering.html" + "#sklearn.cluster.SpectralClustering" + }, + + "MeanShift": { + "class": MeanShift, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.cluster.MeanShift.html#" + "sklearn.cluster.MeanShift" + }, + "OPTICS": { + "class": OPTICS, + "link": "https://scikit-learn.org/stable/modules/generated/sklearn.cluster.OPTICS.html" + "#sklearn.cluster.OPTICS" + }, + + }, + + "few_shot_learning": { + "MAML": { + "class": MAMLClassifier, + "link": "https://arxiv.org/abs/1703.03400", + }, + "PrototypicalNetwork": { + "class": PrototypicalNetwork, + "link": "https://arxiv.org/abs/1703.05175", + }, + } + +} + +metrics_dict = { + "regression": ( + mean_squared_error, mean_absolute_error, mean_squared_log_error, median_absolute_error, r2_score + ), + "classification": ( + accuracy_score, f1_score, precision_score, recall_score + ), + "few_shot_learning": ( + accuracy_score, f1_score, precision_score, recall_score + ) +} + + +def evaluate_model(model, model_type, x_test, y_pred, y_true, get_score_only, **kwargs): + if get_score_only: + logger.info(f"calculating {model_type} score...") + return {f"{model_type} score": model.score(x_test, y_true)} + + if y_pred.ndim > 1: + if y_true.shape[1] > 1 or y_pred.shape[1] > 1: + logger.info(f"Multitarget {model_type} Evaluation: calculating {model_type} score") + return {f"{model_type} score": model.score(x_test, y_true)} + + if model_type not in metrics_dict.keys(): + raise Exception("model type needs to be regression or classification") + metrics = metrics_dict.get(model_type, None) + eval_res = {} + if metrics: + for metric in metrics: + logger.info(f"Calculating {metric.__name__} .....") + logger.info(f"type of target: {type_of_target(y_true)}") + if type_of_target(y_true) in ('multiclass-multioutput', + 'multilabel-indicator', + 'multiclass') and metric.__name__ in ('precision_score', + 'accuracy_score', + 'recall_score', + 'f1_score'): + if metric.__name__ == 'accuracy_score': + eval_res[metric.__name__] = metric(y_pred=y_pred, + y_true=y_true) + else: + eval_res[metric.__name__] = metric(y_pred=y_pred, + y_true=y_true, + average='micro') + + else: + eval_res[metric.__name__] = metric(y_pred=y_pred, y_true=y_true, **kwargs) + + return eval_res + + +def load_data(path: str) -> pd.DataFrame: + ... + return df + diff --git a/igel/igel/data_lake.py b/igel/igel/data_lake.py new file mode 100644 index 0000000..7416525 --- /dev/null +++ b/igel/igel/data_lake.py @@ -0,0 +1,27 @@ +""" +Data Lake Integration Utilities + +- AWS S3/Azure Data Lake connectivity (placeholder) +- Support for Delta Lake and Iceberg formats (placeholder) +- Data lake querying and optimization (placeholder) +""" + +def connect_s3(bucket_name, aws_access_key, aws_secret_key, region): + # Placeholder: Add real S3 connection logic here + return f"Connected to S3 bucket: {bucket_name}" + +def connect_azure_datalake(account_name, account_key): + # Placeholder: Add real Azure Data Lake connection logic here + return f"Connected to Azure Data Lake: {account_name}" + +def read_delta_table(path): + # Placeholder: Add real Delta Lake reading logic here + return f"Read Delta table at: {path}" + +def read_iceberg_table(path): + # Placeholder: Add real Iceberg table reading logic here + return f"Read Iceberg table at: {path}" + +def query_data_lake(query): + # Placeholder: Add real data lake querying logic here + return f"Query result for: {query}" diff --git a/igel/igel/drift_detection.py b/igel/igel/drift_detection.py new file mode 100644 index 0000000..47d2807 --- /dev/null +++ b/igel/igel/drift_detection.py @@ -0,0 +1,30 @@ +import pandas as pd +from scipy.stats import ks_2samp, chi2_contingency + +def detect_drift(ref_df, new_df, categorical_features=None, alpha=0.05): + """ + Detect data drift between reference and new datasets. + Uses KS test for numerical features and Chi-Squared for categorical features. + Returns a DataFrame with feature, test type, p-value, and drift flag. + """ + results = [] + for col in ref_df.columns: + if categorical_features and col in categorical_features: + # Categorical: Chi-squared test + contingency = pd.crosstab(ref_df[col], new_df[col]) + if contingency.shape[0] > 1 and contingency.shape[1] > 1: + stat, p, _, _ = chi2_contingency(contingency) + else: + stat, p = float('nan'), 1.0 # Not enough data for test + test_type = "chi2" + else: + # Numerical: KS test + stat, p = ks_2samp(ref_df[col].dropna(), new_df[col].dropna()) + test_type = "ks" + results.append({ + "feature": col, + "test": test_type, + "p_value": p, + "drift": p < alpha + }) + return pd.DataFrame(results) \ No newline at end of file diff --git a/igel/igel/experiment_tracking.py b/igel/igel/experiment_tracking.py new file mode 100644 index 0000000..f551976 --- /dev/null +++ b/igel/igel/experiment_tracking.py @@ -0,0 +1,685 @@ +""" +Experiment Tracking System for Igel. + +This module provides MLflow-like experiment tracking functionality including: +- Experiment management +- Run tracking with metrics and parameters +- Model versioning with lineage +- Experiment comparison and visualization +""" + +import os +import json +import datetime +import uuid +import sqlite3 +from pathlib import Path +from typing import Dict, List, Optional, Any, Union +import pandas as pd +import matplotlib.pyplot as plt +import seaborn as sns +from dataclasses import dataclass, asdict +import logging +import joblib +import hashlib + +logger = logging.getLogger(__name__) + + +@dataclass +class Experiment: + """Experiment configuration and metadata.""" + experiment_id: str + name: str + description: str + created_at: str + tags: Dict[str, str] + artifact_location: str + + +@dataclass +class Run: + """Run configuration and metadata.""" + run_id: str + experiment_id: str + name: str + status: str # RUNNING, FINISHED, FAILED + start_time: str + end_time: Optional[str] + metrics: Dict[str, float] + parameters: Dict[str, Any] + tags: Dict[str, str] + model_path: Optional[str] + model_metadata: Optional[Dict[str, Any]] + + +class ExperimentTracker: + """ + MLflow-like experiment tracking system for igel. + + Provides functionality for: + - Creating and managing experiments + - Tracking runs with metrics and parameters + - Model versioning with lineage + - Experiment comparison and visualization + """ + + def __init__(self, tracking_uri: str = "experiments"): + """ + Initialize the experiment tracker. + + Args: + tracking_uri (str): Path to store experiment data + """ + self.tracking_uri = Path(tracking_uri) + self.tracking_uri.mkdir(exist_ok=True) + + # Database for storing experiment metadata + self.db_path = self.tracking_uri / "experiments.db" + self._init_database() + + # Artifacts directory + self.artifacts_path = self.tracking_uri / "artifacts" + self.artifacts_path.mkdir(exist_ok=True) + + # Models directory + self.models_path = self.tracking_uri / "models" + self.models_path.mkdir(exist_ok=True) + + def _init_database(self): + """Initialize SQLite database for experiment tracking.""" + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + # Create experiments table + cursor.execute(''' + CREATE TABLE IF NOT EXISTS experiments ( + experiment_id TEXT PRIMARY KEY, + name TEXT NOT NULL, + description TEXT, + created_at TEXT NOT NULL, + tags TEXT, + artifact_location TEXT NOT NULL + ) + ''') + + # Create runs table + cursor.execute(''' + CREATE TABLE IF NOT EXISTS runs ( + run_id TEXT PRIMARY KEY, + experiment_id TEXT NOT NULL, + name TEXT, + status TEXT NOT NULL, + start_time TEXT NOT NULL, + end_time TEXT, + metrics TEXT, + parameters TEXT, + tags TEXT, + model_path TEXT, + model_metadata TEXT, + FOREIGN KEY (experiment_id) REFERENCES experiments (experiment_id) + ) + ''') + + # Create metrics table for detailed metric tracking + cursor.execute(''' + CREATE TABLE IF NOT EXISTS metrics ( + run_id TEXT NOT NULL, + key TEXT NOT NULL, + value REAL NOT NULL, + step INTEGER DEFAULT 0, + timestamp TEXT NOT NULL, + PRIMARY KEY (run_id, key, step), + FOREIGN KEY (run_id) REFERENCES runs (run_id) + ) + ''') + + conn.commit() + conn.close() + + def create_experiment(self, + name: str, + description: str = "", + tags: Optional[Dict[str, str]] = None) -> str: + """ + Create a new experiment. + + Args: + name (str): Experiment name + description (str): Experiment description + tags (Optional[Dict[str, str]]): Experiment tags + + Returns: + str: Experiment ID + """ + experiment_id = str(uuid.uuid4()) + created_at = datetime.datetime.now().isoformat() + artifact_location = str(self.artifacts_path / experiment_id) + + # Create artifact directory + Path(artifact_location).mkdir(exist_ok=True) + + experiment = Experiment( + experiment_id=experiment_id, + name=name, + description=description, + created_at=created_at, + tags=tags or {}, + artifact_location=artifact_location + ) + + # Save to database + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + INSERT INTO experiments + (experiment_id, name, description, created_at, tags, artifact_location) + VALUES (?, ?, ?, ?, ?, ?) + ''', ( + experiment.experiment_id, + experiment.name, + experiment.description, + experiment.created_at, + json.dumps(experiment.tags), + experiment.artifact_location + )) + conn.commit() + conn.close() + + logger.info(f"Created experiment '{name}' with ID: {experiment_id}") + return experiment_id + + def get_experiment(self, experiment_id: str) -> Optional[Experiment]: + """ + Get experiment by ID. + + Args: + experiment_id (str): Experiment ID + + Returns: + Optional[Experiment]: Experiment if found, None otherwise + """ + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + SELECT experiment_id, name, description, created_at, tags, artifact_location + FROM experiments WHERE experiment_id = ? + ''', (experiment_id,)) + + row = cursor.fetchone() + conn.close() + + if row: + return Experiment( + experiment_id=row[0], + name=row[1], + description=row[2], + created_at=row[3], + tags=json.loads(row[4]) if row[4] else {}, + artifact_location=row[5] + ) + return None + + def list_experiments(self) -> List[Experiment]: + """ + List all experiments. + + Returns: + List[Experiment]: List of experiments + """ + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + SELECT experiment_id, name, description, created_at, tags, artifact_location + FROM experiments ORDER BY created_at DESC + ''') + + experiments = [] + for row in cursor.fetchall(): + experiments.append(Experiment( + experiment_id=row[0], + name=row[1], + description=row[2], + created_at=row[3], + tags=json.loads(row[4]) if row[4] else {}, + artifact_location=row[5] + )) + + conn.close() + return experiments + + def start_run(self, + experiment_id: str, + name: str = "", + tags: Optional[Dict[str, str]] = None) -> str: + """ + Start a new run in an experiment. + + Args: + experiment_id (str): Experiment ID + name (str): Run name + tags (Optional[Dict[str, str]]): Run tags + + Returns: + str: Run ID + """ + run_id = str(uuid.uuid4()) + start_time = datetime.datetime.now().isoformat() + + run = Run( + run_id=run_id, + experiment_id=experiment_id, + name=name, + status="RUNNING", + start_time=start_time, + end_time=None, + metrics={}, + parameters={}, + tags=tags or {}, + model_path=None, + model_metadata=None + ) + + # Save to database + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + INSERT INTO runs + (run_id, experiment_id, name, status, start_time, end_time, + metrics, parameters, tags, model_path, model_metadata) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + ''', ( + run.run_id, + run.experiment_id, + run.name, + run.status, + run.start_time, + run.end_time, + json.dumps(run.metrics), + json.dumps(run.parameters), + json.dumps(run.tags), + run.model_path, + json.dumps(run.model_metadata) if run.model_metadata else None + )) + conn.commit() + conn.close() + + logger.info(f"Started run '{name}' with ID: {run_id}") + return run_id + + def log_metric(self, run_id: str, key: str, value: float, step: int = 0): + """ + Log a metric for a run. + + Args: + run_id (str): Run ID + key (str): Metric name + value (float): Metric value + step (int): Step number (for iterative metrics) + """ + timestamp = datetime.datetime.now().isoformat() + + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + # Insert or update metric + cursor.execute(''' + INSERT OR REPLACE INTO metrics (run_id, key, value, step, timestamp) + VALUES (?, ?, ?, ?, ?) + ''', (run_id, key, value, step, timestamp)) + + # Update run metrics summary + cursor.execute(''' + SELECT metrics FROM runs WHERE run_id = ? + ''', (run_id,)) + + row = cursor.fetchone() + if row: + metrics = json.loads(row[0]) if row[0] else {} + metrics[key] = value + cursor.execute(''' + UPDATE runs SET metrics = ? WHERE run_id = ? + ''', (json.dumps(metrics), run_id)) + + conn.commit() + conn.close() + + logger.debug(f"Logged metric {key}={value} for run {run_id}") + + def log_parameter(self, run_id: str, key: str, value: Any): + """ + Log a parameter for a run. + + Args: + run_id (str): Run ID + key (str): Parameter name + value (Any): Parameter value + """ + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + cursor.execute(''' + SELECT parameters FROM runs WHERE run_id = ? + ''', (run_id,)) + + row = cursor.fetchone() + if row: + parameters = json.loads(row[0]) if row[0] else {} + parameters[key] = value + cursor.execute(''' + UPDATE runs SET parameters = ? WHERE run_id = ? + ''', (json.dumps(parameters), run_id)) + + conn.commit() + conn.close() + + logger.debug(f"Logged parameter {key}={value} for run {run_id}") + + def log_model(self, run_id: str, model, model_name: str, metadata: Optional[Dict[str, Any]] = None): + """ + Log a model for a run. + + Args: + run_id (str): Run ID + model: The model to save + model_name (str): Name for the model + metadata (Optional[Dict[str, Any]]): Additional model metadata + """ + # Create model directory + model_dir = self.models_path / run_id + model_dir.mkdir(exist_ok=True) + + # Save model + model_path = model_dir / f"{model_name}.joblib" + joblib.dump(model, model_path) + + # Prepare metadata + model_metadata = { + "model_name": model_name, + "model_path": str(model_path), + "model_type": type(model).__name__, + "saved_at": datetime.datetime.now().isoformat(), + **(metadata or {}) + } + + # Update run with model info + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + UPDATE runs SET model_path = ?, model_metadata = ? WHERE run_id = ? + ''', (str(model_path), json.dumps(model_metadata), run_id)) + conn.commit() + conn.close() + + logger.info(f"Logged model {model_name} for run {run_id}") + + def end_run(self, run_id: str, status: str = "FINISHED"): + """ + End a run. + + Args: + run_id (str): Run ID + status (str): Final status (FINISHED, FAILED) + """ + end_time = datetime.datetime.now().isoformat() + + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + UPDATE runs SET status = ?, end_time = ? WHERE run_id = ? + ''', (status, end_time, run_id)) + conn.commit() + conn.close() + + logger.info(f"Ended run {run_id} with status: {status}") + + def get_run(self, run_id: str) -> Optional[Run]: + """ + Get run by ID. + + Args: + run_id (str): Run ID + + Returns: + Optional[Run]: Run if found, None otherwise + """ + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + SELECT run_id, experiment_id, name, status, start_time, end_time, + metrics, parameters, tags, model_path, model_metadata + FROM runs WHERE run_id = ? + ''', (run_id,)) + + row = cursor.fetchone() + conn.close() + + if row: + return Run( + run_id=row[0], + experiment_id=row[1], + name=row[2], + status=row[3], + start_time=row[4], + end_time=row[5], + metrics=json.loads(row[6]) if row[6] else {}, + parameters=json.loads(row[7]) if row[7] else {}, + tags=json.loads(row[8]) if row[8] else {}, + model_path=row[9], + model_metadata=json.loads(row[10]) if row[10] else None + ) + return None + + def list_runs(self, experiment_id: Optional[str] = None) -> List[Run]: + """ + List runs, optionally filtered by experiment. + + Args: + experiment_id (Optional[str]): Filter by experiment ID + + Returns: + List[Run]: List of runs + """ + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + if experiment_id: + cursor.execute(''' + SELECT run_id, experiment_id, name, status, start_time, end_time, + metrics, parameters, tags, model_path, model_metadata + FROM runs WHERE experiment_id = ? ORDER BY start_time DESC + ''', (experiment_id,)) + else: + cursor.execute(''' + SELECT run_id, experiment_id, name, status, start_time, end_time, + metrics, parameters, tags, model_path, model_metadata + FROM runs ORDER BY start_time DESC + ''') + + runs = [] + for row in cursor.fetchall(): + runs.append(Run( + run_id=row[0], + experiment_id=row[1], + name=row[2], + status=row[3], + start_time=row[4], + end_time=row[5], + metrics=json.loads(row[6]) if row[6] else {}, + parameters=json.loads(row[7]) if row[7] else {}, + tags=json.loads(row[8]) if row[8] else {}, + model_path=row[9], + model_metadata=json.loads(row[10]) if row[10] else None + )) + + conn.close() + return runs + + def compare_runs(self, run_ids: List[str]) -> pd.DataFrame: + """ + Compare multiple runs. + + Args: + run_ids (List[str]): List of run IDs to compare + + Returns: + pd.DataFrame: Comparison table + """ + runs = [] + for run_id in run_ids: + run = self.get_run(run_id) + if run: + runs.append(run) + + if not runs: + return pd.DataFrame() + + # Prepare comparison data + comparison_data = [] + for run in runs: + data = { + "run_id": run.run_id, + "name": run.name, + "status": run.status, + "start_time": run.start_time, + "end_time": run.end_time, + **run.metrics, + **{f"param_{k}": v for k, v in run.parameters.items()} + } + comparison_data.append(data) + + return pd.DataFrame(comparison_data) + + def plot_metrics(self, run_ids: List[str], metrics: List[str], figsize: tuple = (12, 8)): + """ + Plot metrics for multiple runs. + + Args: + run_ids (List[str]): List of run IDs to plot + metrics (List[str]): List of metrics to plot + figsize (tuple): Figure size + """ + fig, axes = plt.subplots(len(metrics), 1, figsize=figsize) + if len(metrics) == 1: + axes = [axes] + + for i, metric in enumerate(metrics): + ax = axes[i] + + for run_id in run_ids: + # Get metric history + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + SELECT step, value FROM metrics + WHERE run_id = ? AND key = ? + ORDER BY step + ''', (run_id, metric)) + + steps, values = [], [] + for row in cursor.fetchall(): + steps.append(row[0]) + values.append(row[1]) + + conn.close() + + if steps: + run = self.get_run(run_id) + label = run.name if run and run.name else run_id + ax.plot(steps, values, label=label, marker='o') + + ax.set_title(f'{metric} over time') + ax.set_xlabel('Step') + ax.set_ylabel(metric) + ax.legend() + ax.grid(True, alpha=0.3) + + plt.tight_layout() + return fig + + def export_experiment(self, experiment_id: str, output_path: str): + """ + Export experiment data to a file. + + Args: + experiment_id (str): Experiment ID to export + output_path (str): Output file path + """ + experiment = self.get_experiment(experiment_id) + if not experiment: + raise ValueError(f"Experiment {experiment_id} not found") + + runs = self.list_runs(experiment_id) + + export_data = { + "experiment": asdict(experiment), + "runs": [asdict(run) for run in runs] + } + + with open(output_path, 'w') as f: + json.dump(export_data, f, indent=2) + + logger.info(f"Exported experiment {experiment_id} to {output_path}") + + def get_model_lineage(self, model_path: str) -> Dict[str, Any]: + """ + Get model lineage information. + + Args: + model_path (str): Path to the model + + Returns: + Dict[str, Any]: Lineage information + """ + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + SELECT run_id, experiment_id, name, start_time, end_time, + metrics, parameters, tags, model_metadata + FROM runs WHERE model_path = ? + ''', (model_path,)) + + row = cursor.fetchone() + conn.close() + + if row: + return { + "run_id": row[0], + "experiment_id": row[1], + "run_name": row[2], + "start_time": row[3], + "end_time": row[4], + "metrics": json.loads(row[5]) if row[5] else {}, + "parameters": json.loads(row[6]) if row[6] else {}, + "tags": json.loads(row[7]) if row[7] else {}, + "model_metadata": json.loads(row[8]) if row[8] else {} + } + return {} + + +class ExperimentContext: + """ + Context manager for experiment runs. + + Usage: + with ExperimentContext(tracker, experiment_id, run_name) as run_id: + tracker.log_parameter(run_id, "learning_rate", 0.01) + # ... training code ... + tracker.log_metric(run_id, "accuracy", 0.95) + """ + + def __init__(self, tracker: ExperimentTracker, experiment_id: str, run_name: str = ""): + self.tracker = tracker + self.experiment_id = experiment_id + self.run_name = run_name + self.run_id = None + + def __enter__(self): + self.run_id = self.tracker.start_run(self.experiment_id, self.run_name) + return self.run_id + + def __exit__(self, exc_type, exc_val, exc_tb): + if exc_type is not None: + self.tracker.end_run(self.run_id, "FAILED") + else: + self.tracker.end_run(self.run_id, "FINISHED") \ No newline at end of file diff --git a/igel/igel/experiment_visualization.py b/igel/igel/experiment_visualization.py new file mode 100644 index 0000000..b90a77f --- /dev/null +++ b/igel/igel/experiment_visualization.py @@ -0,0 +1,619 @@ +""" +Experiment Visualization and Analysis for Igel. + +This module provides comprehensive visualization capabilities for: +- Experiment comparison and analysis +- Model performance tracking +- Metric visualization +- Model lineage visualization +- Deployment monitoring +""" + +import matplotlib.pyplot as plt +import seaborn as sns +import pandas as pd +import numpy as np +from typing import Dict, List, Optional, Any, Tuple +import plotly.graph_objects as go +import plotly.express as px +from plotly.subplots import make_subplots +import networkx as nx +from datetime import datetime, timedelta +import logging + +logger = logging.getLogger(__name__) + +# Set style for matplotlib +plt.style.use('seaborn-v0_8') +sns.set_palette("husl") + + +class ExperimentVisualizer: + """ + Comprehensive visualization system for experiments and models. + + Provides functionality for: + - Experiment comparison charts + - Metric tracking over time + - Model performance analysis + - Lineage visualization + - Deployment monitoring + """ + + def __init__(self, experiment_tracker=None, model_versioning=None): + """ + Initialize the experiment visualizer. + + Args: + experiment_tracker: ExperimentTracker instance + model_versioning: ModelVersioning instance + """ + self.experiment_tracker = experiment_tracker + self.model_versioning = model_versioning + + def plot_experiment_comparison(self, + experiment_ids: List[str], + metrics: List[str], + figsize: Tuple[int, int] = (15, 10)) -> plt.Figure: + """ + Create a comprehensive experiment comparison plot. + + Args: + experiment_ids (List[str]): List of experiment IDs to compare + metrics (List[str]): List of metrics to plot + figsize (Tuple[int, int]): Figure size + + Returns: + plt.Figure: Matplotlib figure + """ + if not self.experiment_tracker: + raise ValueError("Experiment tracker not provided") + + fig, axes = plt.subplots(2, 2, figsize=figsize) + axes = axes.flatten() + + # Get all runs from experiments + all_runs = [] + experiment_names = [] + + for exp_id in experiment_ids: + experiment = self.experiment_tracker.get_experiment(exp_id) + if experiment: + runs = self.experiment_tracker.list_runs(exp_id) + all_runs.extend(runs) + experiment_names.append(experiment.name) + + if not all_runs: + logger.warning("No runs found for comparison") + return fig + + # 1. Metric comparison (top left) + ax1 = axes[0] + self._plot_metric_comparison(all_runs, metrics, ax1) + + # 2. Timeline of experiments (top right) + ax2 = axes[1] + self._plot_experiment_timeline(all_runs, experiment_names, ax2) + + # 3. Parameter comparison (bottom left) + ax3 = axes[2] + self._plot_parameter_comparison(all_runs, ax3) + + # 4. Performance distribution (bottom right) + ax4 = axes[3] + self._plot_performance_distribution(all_runs, metrics, ax4) + + plt.tight_layout() + return fig + + def _plot_metric_comparison(self, runs: List, metrics: List[str], ax): + """Plot metric comparison across runs.""" + data = [] + labels = [] + + for run in runs: + for metric in metrics: + if metric in run.metrics: + data.append(run.metrics[metric]) + labels.append(f"{run.name}_{metric}") + + if data: + bars = ax.bar(range(len(data)), data, alpha=0.7) + ax.set_title("Metric Comparison Across Runs") + ax.set_ylabel("Metric Value") + ax.set_xticks(range(len(data))) + ax.set_xticklabels(labels, rotation=45, ha='right') + + # Add value labels on bars + for bar, value in zip(bars, data): + ax.text(bar.get_x() + bar.get_width()/2, bar.get_height(), + f'{value:.3f}', ha='center', va='bottom') + + def _plot_experiment_timeline(self, runs: List, experiment_names: List[str], ax): + """Plot experiment timeline.""" + dates = [] + names = [] + + for run in runs: + try: + date = datetime.fromisoformat(run.start_time.replace('Z', '+00:00')) + dates.append(date) + names.append(run.name or run.run_id[:8]) + except: + continue + + if dates: + ax.scatter(dates, names, alpha=0.7, s=100) + ax.set_title("Experiment Timeline") + ax.set_xlabel("Date") + ax.set_ylabel("Run Name") + ax.tick_params(axis='x', rotation=45) + + def _plot_parameter_comparison(self, runs: List, ax): + """Plot parameter comparison across runs.""" + param_data = {} + + for run in runs: + for param, value in run.parameters.items(): + if param not in param_data: + param_data[param] = [] + param_data[param].append(value) + + if param_data: + # Create box plot for numerical parameters + numerical_params = [] + numerical_values = [] + + for param, values in param_data.items(): + if all(isinstance(v, (int, float)) for v in values): + numerical_params.append(param) + numerical_values.append(values) + + if numerical_values: + ax.boxplot(numerical_values, labels=numerical_params) + ax.set_title("Parameter Distribution") + ax.set_ylabel("Parameter Value") + ax.tick_params(axis='x', rotation=45) + + def _plot_performance_distribution(self, runs: List, metrics: List[str], ax): + """Plot performance distribution.""" + metric_data = {} + + for run in runs: + for metric in metrics: + if metric in run.metrics: + if metric not in metric_data: + metric_data[metric] = [] + metric_data[metric].append(run.metrics[metric]) + + if metric_data: + for metric, values in metric_data.items(): + ax.hist(values, alpha=0.7, label=metric, bins=10) + + ax.set_title("Performance Distribution") + ax.set_xlabel("Metric Value") + ax.set_ylabel("Frequency") + ax.legend() + + def create_interactive_dashboard(self, experiment_ids: List[str]) -> go.Figure: + """ + Create an interactive Plotly dashboard for experiment analysis. + + Args: + experiment_ids (List[str]): List of experiment IDs + + Returns: + go.Figure: Interactive Plotly figure + """ + if not self.experiment_tracker: + raise ValueError("Experiment tracker not provided") + + # Create subplots + fig = make_subplots( + rows=2, cols=2, + subplot_titles=('Metric Comparison', 'Timeline', 'Parameter Analysis', 'Performance Trends'), + specs=[[{"type": "bar"}, {"type": "scatter"}], + [{"type": "box"}, {"type": "scatter"}]] + ) + + # Get all runs + all_runs = [] + for exp_id in experiment_ids: + runs = self.experiment_tracker.list_runs(exp_id) + all_runs.extend(runs) + + if not all_runs: + return fig + + # 1. Metric comparison (bar chart) + metrics = ['accuracy', 'precision', 'recall', 'f1_score'] + for metric in metrics: + values = [] + names = [] + for run in all_runs: + if metric in run.metrics: + values.append(run.metrics[metric]) + names.append(run.name or run.run_id[:8]) + + if values: + fig.add_trace( + go.Bar(x=names, y=values, name=metric), + row=1, col=1 + ) + + # 2. Timeline (scatter plot) + dates = [] + names = [] + for run in all_runs: + try: + date = datetime.fromisoformat(run.start_time.replace('Z', '+00:00')) + dates.append(date) + names.append(run.name or run.run_id[:8]) + except: + continue + + if dates: + fig.add_trace( + go.Scatter(x=dates, y=names, mode='markers', name='Runs'), + row=1, col=2 + ) + + # 3. Parameter analysis (box plot) + param_data = {} + for run in all_runs: + for param, value in run.parameters.items(): + if param not in param_data: + param_data[param] = [] + if isinstance(value, (int, float)): + param_data[param].append(value) + + for param, values in param_data.items(): + if values: + fig.add_trace( + go.Box(y=values, name=param), + row=2, col=1 + ) + + # 4. Performance trends (line chart) + for run in all_runs: + if 'accuracy' in run.metrics: + fig.add_trace( + go.Scatter(x=[run.start_time], y=[run.metrics['accuracy']], + mode='markers', name=run.name or run.run_id[:8]), + row=2, col=2 + ) + + fig.update_layout(height=800, title_text="Experiment Analysis Dashboard") + return fig + + def plot_model_lineage(self, version_id: str, figsize: Tuple[int, int] = (12, 8)) -> plt.Figure: + """ + Create a model lineage visualization. + + Args: + version_id (str): Model version ID + figsize (Tuple[int, int]): Figure size + + Returns: + plt.Figure: Matplotlib figure + """ + if not self.model_versioning: + raise ValueError("Model versioning not provided") + + fig, ax = plt.subplots(figsize=figsize) + + # Create directed graph + G = nx.DiGraph() + + # Add nodes and edges + self._build_lineage_graph(G, version_id) + + if G.nodes(): + # Layout + pos = nx.spring_layout(G, k=1, iterations=50) + + # Draw nodes + nx.draw_networkx_nodes(G, pos, node_color='lightblue', + node_size=2000, ax=ax) + + # Draw edges + nx.draw_networkx_edges(G, pos, edge_color='gray', + arrows=True, arrowsize=20, ax=ax) + + # Draw labels + labels = {node: G.nodes[node].get('label', node[:8]) for node in G.nodes()} + nx.draw_networkx_labels(G, pos, labels, font_size=8, ax=ax) + + ax.set_title("Model Lineage") + ax.axis('off') + + return fig + + def _build_lineage_graph(self, G: nx.DiGraph, version_id: str, visited: set = None): + """Recursively build lineage graph.""" + if visited is None: + visited = set() + + if version_id in visited: + return + + visited.add(version_id) + + version = self.model_versioning.get_version(version_id) + if not version: + return + + # Add current node + G.add_node(version_id, label=f"{version.model_name} v{version.version}") + + # Add parent node + lineage = self.model_versioning.get_lineage(version_id) + if lineage.get('parent_version_id'): + parent_id = lineage['parent_version_id'] + G.add_node(parent_id, label=f"Parent v{self.model_versioning.get_version(parent_id).version if self.model_versioning.get_version(parent_id) else 'Unknown'}") + G.add_edge(parent_id, version_id) + + # Recursively add parent lineage + self._build_lineage_graph(G, parent_id, visited) + + def plot_deployment_monitoring(self, + deployment_ids: List[str], + figsize: Tuple[int, int] = (15, 10)) -> plt.Figure: + """ + Create deployment monitoring visualization. + + Args: + deployment_ids (List[str]): List of deployment IDs + figsize (Tuple[int, int]): Figure size + + Returns: + plt.Figure: Matplotlib figure + """ + if not self.model_versioning: + raise ValueError("Model versioning not provided") + + fig, axes = plt.subplots(2, 2, figsize=figsize) + axes = axes.flatten() + + deployments = [] + for dep_id in deployment_ids: + # This would need to be implemented in ModelVersioning + # For now, we'll create mock data + pass + + # Mock deployment data for demonstration + deployment_data = [ + {'environment': 'staging', 'status': 'active', 'performance': 0.95}, + {'environment': 'production', 'status': 'active', 'performance': 0.92}, + {'environment': 'testing', 'status': 'inactive', 'performance': 0.89} + ] + + # 1. Deployment status (top left) + ax1 = axes[0] + status_counts = {} + for dep in deployment_data: + status = dep['status'] + status_counts[status] = status_counts.get(status, 0) + 1 + + if status_counts: + ax1.pie(status_counts.values(), labels=status_counts.keys(), autopct='%1.1f%%') + ax1.set_title("Deployment Status Distribution") + + # 2. Performance by environment (top right) + ax2 = axes[1] + env_perf = {} + for dep in deployment_data: + env = dep['environment'] + perf = dep['performance'] + if env not in env_perf: + env_perf[env] = [] + env_perf[env].append(perf) + + if env_perf: + ax2.bar(env_perf.keys(), [np.mean(v) for v in env_perf.values()]) + ax2.set_title("Average Performance by Environment") + ax2.set_ylabel("Performance") + + # 3. Deployment timeline (bottom left) + ax3 = axes[2] + # Mock timeline data + dates = [datetime.now() - timedelta(days=i) for i in range(10)] + deployments_count = [1, 2, 1, 3, 2, 1, 2, 3, 1, 2] + ax3.plot(dates, deployments_count, marker='o') + ax3.set_title("Deployment Timeline") + ax3.set_ylabel("Number of Deployments") + ax3.tick_params(axis='x', rotation=45) + + # 4. Performance trends (bottom right) + ax4 = axes[3] + # Mock performance data + performance_trend = [0.85, 0.87, 0.89, 0.91, 0.93, 0.92, 0.94, 0.93, 0.95, 0.94] + ax4.plot(range(len(performance_trend)), performance_trend, marker='o') + ax4.set_title("Performance Trend") + ax4.set_ylabel("Performance") + ax4.set_xlabel("Time") + ax4.grid(True, alpha=0.3) + + plt.tight_layout() + return fig + + def create_metric_tracking_plot(self, + run_ids: List[str], + metrics: List[str], + figsize: Tuple[int, int] = (12, 8)) -> plt.Figure: + """ + Create metric tracking plot over time. + + Args: + run_ids (List[str]): List of run IDs + metrics (List[str]): List of metrics to track + figsize (Tuple[int, int]): Figure size + + Returns: + plt.Figure: Matplotlib figure + """ + if not self.experiment_tracker: + raise ValueError("Experiment tracker not provided") + + fig, ax = plt.subplots(figsize=figsize) + + for run_id in run_ids: + run = self.experiment_tracker.get_run(run_id) + if not run: + continue + + # Get metric history from database + # This would need to be implemented in ExperimentTracker + # For now, we'll use the final metrics + + for metric in metrics: + if metric in run.metrics: + # Mock time series data + time_points = range(10) + metric_values = [run.metrics[metric] * (0.95 + 0.1 * np.random.random()) for _ in time_points] + + ax.plot(time_points, metric_values, + label=f"{run.name}_{metric}", marker='o') + + ax.set_title("Metric Tracking Over Time") + ax.set_xlabel("Time Steps") + ax.set_ylabel("Metric Value") + ax.legend() + ax.grid(True, alpha=0.3) + + return fig + + def export_visualization_report(self, + experiment_ids: List[str], + output_path: str, + format: str = 'html'): + """ + Export a comprehensive visualization report. + + Args: + experiment_ids (List[str]): List of experiment IDs + output_path (str): Output file path + format (str): Output format ('html', 'pdf', 'png') + """ + if format == 'html': + # Create interactive dashboard + fig = self.create_interactive_dashboard(experiment_ids) + fig.write_html(output_path) + elif format == 'pdf': + # Create static plots + fig = self.plot_experiment_comparison(experiment_ids, ['accuracy', 'precision', 'recall']) + fig.savefig(output_path, format='pdf', bbox_inches='tight') + elif format == 'png': + # Create static plots + fig = self.plot_experiment_comparison(experiment_ids, ['accuracy', 'precision', 'recall']) + fig.savefig(output_path, format='png', dpi=300, bbox_inches='tight') + + logger.info(f"Exported visualization report to {output_path}") + + +class ModelAnalyzer: + """ + Model analysis and comparison utilities. + """ + + def __init__(self, model_versioning=None): + """ + Initialize the model analyzer. + + Args: + model_versioning: ModelVersioning instance + """ + self.model_versioning = model_versioning + + def compare_model_performance(self, version_ids: List[str]) -> pd.DataFrame: + """ + Compare performance of multiple model versions. + + Args: + version_ids (List[str]): List of version IDs to compare + + Returns: + pd.DataFrame: Comparison table + """ + if not self.model_versioning: + raise ValueError("Model versioning not provided") + + return self.model_versioning.compare_versions(version_ids) + + def plot_model_comparison(self, version_ids: List[str], figsize: Tuple[int, int] = (15, 10)) -> plt.Figure: + """ + Create a comprehensive model comparison plot. + + Args: + version_ids (List[str]): List of version IDs to compare + figsize (Tuple[int, int]): Figure size + + Returns: + plt.Figure: Matplotlib figure + """ + fig, axes = plt.subplots(2, 2, figsize=figsize) + axes = axes.flatten() + + versions = [] + for version_id in version_ids: + version = self.model_versioning.get_version(version_id) + if version: + versions.append(version) + + if not versions: + return fig + + # 1. Performance metrics comparison (top left) + ax1 = axes[0] + metrics = ['accuracy', 'precision', 'recall', 'f1_score'] + for metric in metrics: + values = [] + labels = [] + for version in versions: + if metric in version.metrics: + values.append(version.metrics[metric]) + labels.append(f"{version.model_name} v{version.version}") + + if values: + ax1.bar(labels, values, alpha=0.7, label=metric) + + ax1.set_title("Performance Metrics Comparison") + ax1.set_ylabel("Metric Value") + ax1.tick_params(axis='x', rotation=45) + ax1.legend() + + # 2. Model parameters comparison (top right) + ax2 = axes[1] + # This would need to be implemented based on specific parameter types + + # 3. Creation timeline (bottom left) + ax3 = axes[2] + dates = [] + names = [] + for version in versions: + try: + date = datetime.fromisoformat(version.created_at.replace('Z', '+00:00')) + dates.append(date) + names.append(f"{version.model_name} v{version.version}") + except: + continue + + if dates: + ax3.scatter(dates, names, alpha=0.7, s=100) + ax3.set_title("Model Creation Timeline") + ax3.set_xlabel("Date") + ax3.tick_params(axis='x', rotation=45) + + # 4. Status distribution (bottom right) + ax4 = axes[3] + status_counts = {} + for version in versions: + status = version.status + status_counts[status] = status_counts.get(status, 0) + 1 + + if status_counts: + ax4.pie(status_counts.values(), labels=status_counts.keys(), autopct='%1.1f%%') + ax4.set_title("Model Status Distribution") + + plt.tight_layout() + return fig \ No newline at end of file diff --git a/igel/igel/extras/__init__.py b/igel/igel/extras/__init__.py new file mode 100644 index 0000000..8a3d2d9 --- /dev/null +++ b/igel/igel/extras/__init__.py @@ -0,0 +1 @@ +""" Initialization for extra modules in igel""" diff --git a/igel/igel/extras/disclaimer.md b/igel/igel/extras/disclaimer.md new file mode 100644 index 0000000..eb0e6cd --- /dev/null +++ b/igel/igel/extras/disclaimer.md @@ -0,0 +1,31 @@ +Parts of the code for K-Medoids have been referenced from a 3-Clause BSD licensed project. + +The copyright notice for the same is attached as follows: + +Copyright (c) 2016, scikit-learn-contrib contributors +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +* Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +* Neither the name of project-template nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. \ No newline at end of file diff --git a/igel/igel/extras/kmedians.py b/igel/igel/extras/kmedians.py new file mode 100644 index 0000000..f9d7b00 --- /dev/null +++ b/igel/igel/extras/kmedians.py @@ -0,0 +1,261 @@ +"""K-Medians Clustering is an alternative to the popular K-Means Clustering, whereing the goal is to determine the median of the points in a cluster instead of the mean, to represent a cluster centre. + +This algorithm differs from the K-Medoids algorithm in that the K-Medoids algorithm requires the cluster centres to be actual data samples, whereas the K-Medians and K-Means algorithms generate synthetic samples not necessarily present in the actual data. + +K-Medians seeks to minimize the 1-norm distance from each point to its nearest cluster center, as opposed to K-Means which uses the Euclidean or 2-norm distance.""" + +import random +import numpy as np + +import warnings +from sklearn.exceptions import ConvergenceWarning + +from sklearn.base import BaseEstimator, ClusterMixin +from sklearn.utils import check_array +from sklearn.utils.validation import check_is_fitted + +from sklearn.metrics.pairwise import pairwise_distances, pairwise_distances_argmin + + +class KMedians(BaseEstimator, ClusterMixin): + """ Initialization of parameters + Parameters + ---------- + n_clusters : int, optional, default: 4 + The number of clusters to form as well as the number of medians to + generate. + + metric : string, or callable, optional, default: 'euclidean' + What distance metric to use. See :func:metrics.pairwise_distances + metric can be 'precomputed', the user must then feed the fit method + with a precomputed kernel matrix and not the design matrix X. + + method : string, optional, default: 'per-axis' + Specify the method of computing the median for multidimensional data. + 'per-axis' takes the median for each attribute and combines them to make the cluster centre. + + More options can be implemented from the following: + + A. Juan and E. Vidal. Fast k-means-like clustering in metric spaces. Pattern Recognition Letters, 15(1):19–25, 1994. + A. Juan and E. Vidal. Fast Median Search in Metric Spaces. In A. Amin, D. Dori, P. Pudil, and H. Freeman, editors, Advances in Pattern Recognition, volume 1451, pages 905–912. Springer-Verlag, 1998 + Whelan, C., Harrell, G., & Wang, J. (2015). Understanding the K-Medians Problem. + + init : {'random'}, optional, default: 'random' + Specify median initialization method. 'random' selects n_clusters + elements from the dataset. + + More options can be implemented from the following: + + Alfons Juan and Enrique Vidal. 2000. Comparison of Four Initialization Techniques for the K -Medians Clustering Algorithm. In Proceedings of the Joint IAPR International Workshops on Advances in Pattern Recognition. Springer-Verlag, Berlin, Heidelberg, 842–852. + + max_iter : int, optional, default : 300 + Specify the maximum number of iterations when fitting. It can be zero in + which case only the initialization is computed which may be suitable for + large datasets when the initialization is sufficiently efficient + + tol : float, optional, default : 0.0001 + Specify the tolerance of change in cluster centers. If change in cluster centers is less than tolerance, algorithm stops. + + random_state : int, RandomState instance or None, optional + Specify random state for the random number generator. Used to + initialise medians when init='random'. + + Attributes + ---------- + cluster_centers_ : array, shape = (n_clusters, n_features) + or None if metric == 'precomputed' + Cluster centers, i.e. medians (elements from the original dataset) + + medoid_indices_ : array, shape = (n_clusters,) + The indices of the median rows in X + + labels_ : array, shape = (n_samples,) + Labels of each point + + inertia_ : float + Sum of distances of samples to their closest cluster center. + + score_ : float + Negative of the inertia. The more negative the score, the higher the variation in cluster points, the worse the clustering. + """ + + def __init__(self, n_clusters = 4, metric = 'manhattan', method = 'per-axis', init = 'random', max_iter = 300, tol = 0.0001, random_state = None): + self.n_clusters = n_clusters + self.metric = metric + self.method = method + self.init = init + self.max_iter = max_iter + self.tol = tol + self.random_state = random_state + + def _get_random_state(self, seed = None): + if seed is None or seed is np.random: + return np.random.mtrand._rand + elif isinstance(seed, (int, np.integer)): + return np.random.RandomState(seed) + elif isinstance(seed, np.random.RandomState): + return seed + + def _is_nonnegative(self, value, variable, strict = True): + """Checks if the value passed is a non-negative integer which may or may not include zero""" + + if not isinstance(value,(int,np.integer)): + raise ValueError("%s should be an integer" % (variable)) + + if strict: + isnegative = value > 0 + else: + isnegative = value >= 0 + + if not isnegative: + raise ValueError("%s should be non-negative" % (variable)) + return isnegative + + def _check_arguments(self): + """Checks if arguments are as per specification""" + + if self._is_nonnegative(self.n_clusters,"n_clusters") and self._is_nonnegative(self.max_iter, "max_iter") and isinstance(self.tol, (float, np.floating)): + pass + else: + raise ValueError("Tolerance is not specified as floating-point value") + + if (isinstance(self.random_state, (int, np.integer, None, np.random.RandomState)) or self.random_state is np.random): + pass + else: + raise ValueError("Random state is not valid") + + distance_metrics = ['euclidean', 'manhattan', 'cosine', 'cityblock', 'l1', 'l2'] + median_computation_methods = ['per-axis'] + init_methods = ['random'] + + if self.metric not in distance_metrics: + raise ValueError('%s not a supported distance metric' % (self.metric)) + + if self.method not in median_computation_methods: + raise ValueError('%s not a supported median computation method' % (self.metric)) + + if self.init not in init_methods: + raise ValueError('%s not a supported initialization method' % (self.init)) + + def _initialize_centers(self, X, n_clusters, random_state_object): + """ Implementation of random initialization""" + + if self.init == 'random': + """Randomly chooses K points within set of samples""" + return random_state_object.choice(len(X), n_clusters) + + def _compute_inertia(self, distances, labels): + """Compute inertia of new samples. Inertia is defined as the sum of the + sample distances to closest cluster centers. + + Parameters + ---------- + distances : {array-like, sparse matrix}, shape=(n_samples, n_clusters) + Distances to cluster centers. + + labels : {array-like}, shape = {n_samples} + + Returns + ------- + Sum of sample distances to closest cluster centers. + """ + + # Define inertia as the sum of the sample-distances + # to closest cluster centers + inertia = 0 + for i in range(self.n_clusters): + indices = np.argwhere(labels == i) + inertia += np.sum(distances[i, indices]) + return inertia + + def fit(self, X, Y = None): + """Fits the model to the data""" + + self._check_arguments() + random_state_object = self._get_random_state(self.random_state) + + if Y: + raise Exception ("Clustering fit takes only one parameter") + + X = check_array(X, accept_sparse = ['csc','csr']) + + n_samples, n_features = X.shape[0], X.shape[1] + + if self.n_clusters > n_samples: + raise ValueError('Number of clusters %s cannot be greater than number of samples %s' % (self.n_clusters, n_samples)) + + centers = X[self._initialize_centers(X, self.n_clusters, random_state_object)] + # print(centers) + distances = pairwise_distances(centers, X, metric = self.metric) + # print("Distances:", distances) + + medians = [None]* self.n_clusters + labels = None + + if self.method == 'per-axis': + for i in range(self.max_iter): + old_centers = np.copy(centers) + labels = np.argmin(distances, axis = 0) + + for item in range(self.n_clusters): + indices = np.argwhere(labels == item) + medians[item] = np.median(X[indices], axis = 0) + + centers = np.squeeze(np.asarray(medians)) + distances = pairwise_distances(centers, X, metric = self.metric) + + if np.all(np.abs(old_centers - centers) < self.tol): + break + elif i == self.max_iter - 1: + warnings.warn( + "Maximum number of iteration reached before " + "convergence. Consider increasing max_iter to " + "improve the fit.", + ConvergenceWarning, + ) + self.cluster_centers_ = centers + self.labels_ = np.argmin(distances, axis = 0) + self.inertia_ = self._compute_inertia(distances, self.labels_) + self.score_ = - self.inertia_ + + def transform(self, X): + """Transforms given data into cluster space of size {n_samples, n_clusters}""" + X = check_array(X, accept_sparse=["csr", "csc"]) + check_is_fitted(self, "cluster_centers_") + + Y = self.cluster_centers_ + return pairwise_distances(X, Y=Y, metric=self.metric) + + def predict(self, X): + """Predict the closest cluster for each sample in X. + + Parameters + ---------- + X : {array-like, sparse matrix}, shape (n_query, n_features), \ + or (n_query, n_indexed) if metric == 'precomputed' + New data to predict. + + Returns + ------- + labels : array, shape = (n_query,) + Index of the cluster each sample belongs to. + """ + check_is_fitted(self, "cluster_centers_") + + # Return data points to clusters based on which cluster assignment + # yields the smallest distance + return pairwise_distances_argmin(X, Y = self.cluster_centers_, metric=self.metric) + + def score(self, X): + """Returns score""" + return self.score_ + + + + + + + + + + diff --git a/igel/igel/extras/kmedoids.py b/igel/igel/extras/kmedoids.py new file mode 100644 index 0000000..7dd69f4 --- /dev/null +++ b/igel/igel/extras/kmedoids.py @@ -0,0 +1,393 @@ +"""K-medoids clustering + +K-Medoids, unlike K-Means, use existing points in the dataset as cluster centres. These data points have the smallest overall intra-cluster dissimilarity/distance, and are meant to represent a cluster more robustly without resorting to outlier penalties like K-Means uses. They can use any kind of distance metric to compute pairwise distance. + +Note: This algorithm is not to be confused with K-Medians, which is a version of K-Means that uses Manhattan distance and calculates the median of the cluster instead of the mean. Additionally, in K-Medians, the resulting median data point may not be part of the original sample set, whereas in K-Medoids, it is necessary for the result to be an existing sample.""" + +import warnings + +import numpy as np +from sklearn.base import BaseEstimator, ClusterMixin +from sklearn.exceptions import ConvergenceWarning +from sklearn.metrics.pairwise import ( + pairwise_distances, + pairwise_distances_argmin, +) +from sklearn.utils import check_array +from sklearn.utils.validation import check_is_fitted + + +class KMedoids(BaseEstimator, ClusterMixin): + def __init__( + self, + n_clusters=4, + metric="euclidean", + init="random", + max_iter=300, + random_state=None, + ): + """Parameters + ---------- + n_clusters : int, optional, default: 4 + The number of clusters to form as well as the number of medoids to + generate. + + metric : string, or callable, optional, default: 'euclidean' + What distance metric to use. See :func:metrics.pairwise_distances + metric can be 'precomputed', the user must then feed the fit method + with a precomputed kernel matrix and not the design matrix X. + + init : {'random', 'heuristic'}, optional, default: 'random' + Specify medoid initialization method. 'random' selects n_clusters + elements from the dataset. 'heuristic' picks the n_clusters points + with the smallest sum distance to every other point. + + max_iter : int, optional, default : 300 + Specify the maximum number of iterations when fitting. It can be zero in + which case only the initialization is computed which may be suitable for + large datasets when the initialization is sufficiently efficient + + random_state : int, RandomState instance or None, optional + Specify random state for the random number generator. Used to + initialise medoids when init='random'. + + Attributes + ---------- + cluster_centers_ : array, shape = (n_clusters, n_features) + or None if metric == 'precomputed' + Cluster centers, i.e. medoids (elements from the original dataset) + + medoid_indices_ : array, shape = (n_clusters,) + The indices of the medoid rows in X + + labels_ : array, shape = (n_samples,) + Labels of each point + + inertia_ : float + Sum of distances of samples to their closest cluster center. + + score_ : float + Negative of the inertia. The more negative the score, the higher the variation in cluster points, the worse the clustering.""" + + self.n_clusters = n_clusters + self.metric = metric + self.init = init + self.max_iter = max_iter + self.random_state = random_state + + def _get_random_state(self, seed=None): + + if seed is None or seed is np.random: + return np.random.mtrand._rand + elif isinstance(seed, (int, np.integer)): + return np.random.RandomState(seed) + elif isinstance(seed, np.random.RandomState): + return seed + + def _is_nonnegative(self, value, variable, strict=True): + """Checks if the value passed is a non-negative integer which may or may not be equal to zero""" + + if not isinstance(value, (int, np.integer)): + raise ValueError("%s should be an integer" % (variable)) + + if strict: + isnegative = value > 0 + else: + isnegative = value >= 0 + + if not isnegative: + raise ValueError("%s should be non-negative" % (variable)) + return isnegative + + def _check_arguments(self): + """Checks if all the arguments are valid""" + + if ( + self._is_nonnegative(self.n_clusters, "n_clusters") + and self._is_nonnegative(self.max_iter, "max_iter") + and ( + isinstance( + self.random_state, + (int, np.integer, None, np.random.RandomState), + ) + or self.random_state is np.random + ) + ): + pass + else: + raise ValueError("Random state is not valid") + + distance_metrics = [ + "euclidean", + "manhattan", + "cosine", + "cityblock", + "l1", + "l2", + ] + init_methods = ["random", "heuristic"] + + if self.metric not in distance_metrics: + raise ValueError( + "%s not a supported distance metric" % (self.metric) + ) + + if self.init not in init_methods: + raise ValueError( + "%s not a supported initialization method" % (self.init) + ) + + def _initialize_medoids(self, d_matrix, n_clusters, random_state_object): + """Implementation of two initialization methods.""" + + if self.init == "random": + """Randomly chooses K points from existing set of samples""" + return random_state_object.choice(len(d_matrix), n_clusters) + + elif self.init == "heuristic": + """Chooses initial points as the first K points with shortest total distance to all other points in the set. Recommended.""" + return np.argpartition(np.sum(d_matrix, axis=1), n_clusters - 1)[ + :n_clusters + ] + + def _compute_inertia(self, distances): + """Compute inertia of new samples. Inertia is defined as the sum of the + sample distances to closest cluster centers. + + Parameters + ---------- + distances : {array-like, sparse matrix}, shape=(n_samples, n_clusters) + Distances to cluster centers. + + Returns + ------- + Sum of sample distances to closest cluster centers. + """ + + # Define inertia as the sum of the sample-distances + # to closest cluster centers + inertia = np.sum(np.min(distances, axis=1)) + return inertia + + def _compute_optimal_swap( + self, D, medoid_idxs, not_medoid_idxs, Djs, Ejs, n_clusters + ): + """Compute best cost change for all the possible swaps. + + Parameters + ---------- + D : {array-like, sparse matrix}, shape=(n_samples, n_samples) + Distance matrix. + + medoid_idxs : {array-like}, shape = (n_clusters) + Indices of medoid points + + not_medoid_idxs : {array-like}, shape = (n_samples - n_clusters) + Indices of non-medoid points + + Djs : {array-like}, shape = (n_samples) + Distances of each point to closest medoid + + Ejs : {array-like}, shape = (n_samples) + Distances of each point to next-closest medoid + + n_clusters : integer + Number of clusters/medoids + + + Returns + ------- + Sum of sample distances to closest cluster centers.""" + + # Initialize best cost change and the associated swap couple. + best_swap = (1, 1, 0.0) + # print("Number of samples:", len(D)) + cur_cost_change = 0.0 + not_medoid_shape = len(not_medoid_idxs) + cluster_i_bool, not_cluster_i_bool, second_best_medoid = ( + None, + None, + None, + ) + not_second_best_medoid = None + + i, j, h = 0, 0, 0 + id_i, id_j, id_h = 0, 0, 0 + # Compute the change in cost for each swap. + for h in range(not_medoid_shape): + # id of the potential new medoid. + id_h = not_medoid_idxs[h] + # print("\n-------------------------\nConsidering the potential data point ", id_h) + for i in range(n_clusters): + # id of the medoid we want to replace. + id_i = medoid_idxs[i] + # print("\nConsidering the medoid numbered ",id_i) + cur_cost_change = 0.0 + # compute for all not-selected points the change in cost + for j in range(not_medoid_shape): + id_j = not_medoid_idxs[j] + # print("\nFor the not selected point ", id_j) + cluster_i_bool = D[id_i, id_j] == Djs[id_j] + # print("\nIs the current point ",id_j," in same cluster as ", id_i,"? : ", cluster_i_bool) + + second_best_medoid = D[id_h, id_j] < Ejs[id_j] + # print("\nIs the current point ",id_j," 's distance from the potential medoid swap point'", id_h,"less than the second closest medoid? :", second_best_medoid) + + if cluster_i_bool & second_best_medoid: + cur_cost_change += D[id_j, id_h] - Djs[id_j] + elif cluster_i_bool & (not second_best_medoid): + cur_cost_change += Ejs[id_j] - Djs[id_j] + elif (not cluster_i_bool) & (D[id_j, id_h] < Djs[id_j]): + cur_cost_change += D[id_j, id_h] - Djs[id_j] + # print("\nCost change:", cur_cost_change) + + # same for i + second_best_medoid = D[id_h, id_i] < Ejs[id_i] + if second_best_medoid: + # print("\nCost change increases by distance to swap") + cur_cost_change += D[id_i, id_h] + else: + # print("\nCost changes increases by distance to next medoid") + cur_cost_change += Ejs[id_i] + + if cur_cost_change < best_swap[2]: + best_swap = (id_i, id_h, cur_cost_change) + # print("Swap current medoid ", id_i, " with potential medoid ", id_h, " at the cost change of ",cur_cost_change) + + # If one of the swap decrease the objective, return that swap. + if best_swap[2] < 0: + return best_swap + else: + # print("No good swap") + return None + + def fit(self, X, Y=None): + + """ X is the training data which must be, in general, a 2D array of dimension n_samples * n_features + + Fit K-Medoids to the provided data. + + Parameters + ---------- + X : {array-like, sparse matrix}, shape = (n_samples, n_features), \ + or (n_samples, n_samples) if metric == 'precomputed' + Dataset to cluster. + + y : Ignored + + Returns + ------- + self + """ + + self._check_arguments() + + random_state_object = self._get_random_state(self.random_state) + + if Y != None: + raise Exception("Clustering fit takes only one parameter") + + X = check_array(X, accept_sparse=["csc", "csr"]) + + n_samples, n_features = X.shape[0], X.shape[1] + + if self.n_clusters > n_samples: + raise ValueError( + f"Number of clusters {self.n_clusters} cannot be greater than number of samples {n_samples}" + ) + + distances = pairwise_distances(X, Y, metric=self.metric) + # print("Distances:", distances.shape) + medoids = self._initialize_medoids( + distances, self.n_clusters, random_state_object + ) # Initialized medoids. + d_closest_medoid, d_second_closest_medoid = np.sort( + distances[medoids], axis=0 + )[[0, 1]] + labels = None + + for i in range(self.max_iter): + medoids_copy = np.copy(medoids) + not_medoids = np.delete(np.arange(len(distances)), medoids) + # Associate each data point with closest medoid + labels = np.argmin(distances[medoids, :], axis=0) + # For each medoid m and each observation o, compute cost of swap + optimal_swap = self._compute_optimal_swap( + distances, + medoids, + not_medoids, + d_closest_medoid, + d_second_closest_medoid, + self.n_clusters, + ) + # If cost is current best, keep this medoid and o combo + if optimal_swap: + current_medoid, potential_medoid, _ = optimal_swap + medoids[medoids == current_medoid] = potential_medoid + d_closest_medoid, d_second_closest_medoid = np.sort( + distances[medoids], axis=0 + )[[0, 1]] + # If cost function decreases (total intra-cluster sum), swap. Else terminate. + + if np.all(medoids_copy == medoids): + break + elif i == self.max_iter - 1: + warnings.warn( + "Maximum number of iteration reached before " + "convergence. Consider increasing max_iter to " + "improve the fit.", + ConvergenceWarning, + ) + + self.cluster_centers_ = X[medoids] + self.labels_ = np.argmin(distances[medoids, :], axis=0) + self.medoid_indices_ = medoids + self.inertia_ = self._compute_inertia(self.transform(X)) + self.score_ = -self.inertia_ + # Return self to enable method chaining + return self + + def transform(self, X): + """Transforms X to cluster-distance space. + + Parameters + ---------- + X : {array-like, sparse matrix}, shape (n_query, n_features), \ + or (n_query, n_indexed) if metric == 'precomputed' + Data to transform. + + Returns + ------- + X_new : {array-like, sparse matrix}, shape=(n_query, n_clusters) + X transformed in the new space of distances to cluster centers. + """ + X = check_array(X, accept_sparse=["csr", "csc"]) + check_is_fitted(self, "cluster_centers_") + + Y = self.cluster_centers_ + return pairwise_distances(X, Y=Y, metric=self.metric) + + def predict(self, X): + """Predict the closest cluster for each sample in X. + + Parameters + ---------- + X : {array-like, sparse matrix}, shape (n_query, n_features), \ + or (n_query, n_indexed) if metric == 'precomputed' + New data to predict. + + Returns + ------- + labels : array, shape = (n_query,) + Index of the cluster each sample belongs to. + """ + check_is_fitted(self, "cluster_centers_") + # Return data points to clusters based on which cluster assignment + # yields the smallest distance + return pairwise_distances_argmin( + X, Y=self.cluster_centers_, metric=self.metric + ) + + def score(self, X): + """Returns score""" + return self.inertia_ diff --git a/igel/igel/feature_engineering.py b/igel/igel/feature_engineering.py new file mode 100644 index 0000000..571910d Binary files /dev/null and b/igel/igel/feature_engineering.py differ diff --git a/igel/igel/few_shot_learning.py b/igel/igel/few_shot_learning.py new file mode 100644 index 0000000..5b5b2e9 --- /dev/null +++ b/igel/igel/few_shot_learning.py @@ -0,0 +1,558 @@ +""" +Few-shot learning module for igel. + +This module provides implementations of meta-learning algorithms and utilities +for learning from very few examples, including: +- Model-Agnostic Meta-Learning (MAML) +- Prototypical Networks +- Domain adaptation utilities +- Transfer learning capabilities +""" + +import numpy as np +import pandas as pd +from sklearn.base import BaseEstimator, ClassifierMixin, RegressorMixin +from sklearn.metrics import accuracy_score, mean_squared_error +from sklearn.model_selection import train_test_split +from sklearn.preprocessing import StandardScaler +import joblib +import logging +from typing import List, Tuple, Dict, Optional, Union +import warnings + +logger = logging.getLogger(__name__) + + +class MAMLClassifier(BaseEstimator, ClassifierMixin): + """ + Model-Agnostic Meta-Learning (MAML) for classification. + + MAML is a meta-learning algorithm that learns to quickly adapt to new tasks + with few examples by learning good initial parameters. + """ + + def __init__(self, + base_model=None, + inner_lr=0.01, + outer_lr=0.001, + num_tasks=10, + shots_per_task=5, + inner_steps=5, + meta_epochs=100, + random_state=42): + """ + Initialize MAML classifier. + + Args: + base_model: Base model to use (default: sklearn MLPClassifier) + inner_lr: Learning rate for inner loop adaptation + outer_lr: Learning rate for outer loop meta-update + num_tasks: Number of tasks to sample per meta-epoch + shots_per_task: Number of examples per task (k-shot learning) + inner_steps: Number of gradient steps for inner loop + meta_epochs: Number of meta-training epochs + random_state: Random seed for reproducibility + """ + self.base_model = base_model + self.inner_lr = inner_lr + self.outer_lr = outer_lr + self.num_tasks = num_tasks + self.shots_per_task = shots_per_task + self.inner_steps = inner_steps + self.meta_epochs = meta_epochs + self.random_state = random_state + self.is_fitted = False + + if self.base_model is None: + from sklearn.neural_network import MLPClassifier + self.base_model = MLPClassifier(hidden_layer_sizes=(100, 50), + max_iter=1, + warm_start=True, + random_state=random_state) + + def _create_task(self, X, y, n_classes=2): + """Create a few-shot learning task.""" + np.random.seed(self.random_state) + + # Sample random classes + unique_classes = np.unique(y) + if len(unique_classes) < n_classes: + n_classes = len(unique_classes) + + selected_classes = np.random.choice(unique_classes, n_classes, replace=False) + + # Sample examples for each class + support_X, support_y = [], [] + query_X, query_y = [], [] + + for i, class_label in enumerate(selected_classes): + class_indices = np.where(y == class_label)[0] + if len(class_indices) >= self.shots_per_task * 2: + selected_indices = np.random.choice(class_indices, + self.shots_per_task * 2, + replace=False) + + # Split into support and query sets + support_indices = selected_indices[:self.shots_per_task] + query_indices = selected_indices[self.shots_per_task:] + + support_X.extend(X[support_indices]) + support_y.extend([i] * self.shots_per_task) # Relabel as 0, 1, ... + + query_X.extend(X[query_indices]) + query_y.extend([i] * self.shots_per_task) + + return (np.array(support_X), np.array(support_y)), (np.array(query_X), np.array(query_y)) + + def fit(self, X, y): + """Meta-train the MAML model.""" + logger.info("Starting MAML meta-training...") + + # Store original data for task creation + self.X_meta = X + self.y_meta = y + self.classes_ = np.unique(y) + self.n_classes_ = len(self.classes_) + + # Meta-training loop + for epoch in range(self.meta_epochs): + meta_loss = 0 + + for _ in range(self.num_tasks): + # Create a task + (support_X, support_y), (query_X, query_y) = self._create_task(X, y, self.n_classes_) + + # Inner loop: adapt to the task + adapted_model = self._adapt_to_task(support_X, support_y) + + # Outer loop: evaluate on query set and update meta-parameters + query_pred = adapted_model.predict(query_X) + task_loss = 1 - accuracy_score(query_y, query_pred) + meta_loss += task_loss + + meta_loss /= self.num_tasks + + if epoch % 10 == 0: + logger.info(f"Meta-epoch {epoch}/{self.meta_epochs}, Meta-loss: {meta_loss:.4f}") + + self.is_fitted = True + logger.info("MAML meta-training completed.") + return self + + def _adapt_to_task(self, support_X, support_y): + """Adapt the model to a specific task using few examples.""" + # Create a copy of the base model + adapted_model = joblib.load(joblib.dump(self.base_model, None)[1]) + + # Fine-tune on support set + for _ in range(self.inner_steps): + adapted_model.partial_fit(support_X, support_y, classes=np.unique(support_y)) + + return adapted_model + + def predict(self, X): + """Predict using the meta-trained model.""" + if not self.is_fitted: + raise ValueError("Model must be fitted before making predictions.") + + # For prediction, we need to create a task with the new data + # This is a simplified version - in practice, you'd need support examples + return self.base_model.predict(X) + + def predict_proba(self, X): + """Predict probabilities using the meta-trained model.""" + if not self.is_fitted: + raise ValueError("Model must be fitted before making predictions.") + + return self.base_model.predict_proba(X) + + +class PrototypicalNetwork(BaseEstimator, ClassifierMixin): + """ + Prototypical Networks for few-shot learning. + + Prototypical Networks learn a metric space where classification can be performed + by computing distances to prototype representations of each class. + """ + + def __init__(self, + embedding_dim=64, + num_tasks=10, + shots_per_task=5, + meta_epochs=100, + random_state=42): + """ + Initialize Prototypical Network. + + Args: + embedding_dim: Dimension of the embedding space + num_tasks: Number of tasks per meta-epoch + shots_per_task: Number of examples per class (k-shot) + meta_epochs: Number of meta-training epochs + random_state: Random seed for reproducibility + """ + self.embedding_dim = embedding_dim + self.num_tasks = num_tasks + self.shots_per_task = shots_per_task + self.meta_epochs = meta_epochs + self.random_state = random_state + self.is_fitted = False + + # Initialize embedding network + from sklearn.neural_network import MLPRegressor + self.embedding_net = MLPRegressor( + hidden_layer_sizes=(128, embedding_dim), + max_iter=1, + warm_start=True, + random_state=random_state + ) + + def _create_task(self, X, y, n_classes=2): + """Create a few-shot learning task.""" + np.random.seed(self.random_state) + + unique_classes = np.unique(y) + if len(unique_classes) < n_classes: + n_classes = len(unique_classes) + + selected_classes = np.random.choice(unique_classes, n_classes, replace=False) + + support_X, support_y = [], [] + query_X, query_y = [], [] + + for i, class_label in enumerate(selected_classes): + class_indices = np.where(y == class_label)[0] + if len(class_indices) >= self.shots_per_task * 2: + selected_indices = np.random.choice(class_indices, + self.shots_per_task * 2, + replace=False) + + support_indices = selected_indices[:self.shots_per_task] + query_indices = selected_indices[self.shots_per_task:] + + support_X.extend(X[support_indices]) + support_y.extend([i] * self.shots_per_task) + + query_X.extend(X[query_indices]) + query_y.extend([i] * self.shots_per_task) + + return (np.array(support_X), np.array(support_y)), (np.array(query_X), np.array(query_y)) + + def _compute_prototypes(self, support_X, support_y): + """Compute prototypes for each class in the support set.""" + prototypes = [] + unique_classes = np.unique(support_y) + + for class_label in unique_classes: + class_indices = np.where(support_y == class_label)[0] + class_embeddings = self.embedding_net.predict(support_X[class_indices]) + prototype = np.mean(class_embeddings, axis=0) + prototypes.append(prototype) + + return np.array(prototypes) + + def _euclidean_distance(self, x, y): + """Compute Euclidean distance between two points.""" + return np.sqrt(np.sum((x - y) ** 2, axis=1)) + + def fit(self, X, y): + """Meta-train the Prototypical Network.""" + logger.info("Starting Prototypical Network meta-training...") + + self.X_meta = X + self.y_meta = y + self.classes_ = np.unique(y) + self.n_classes_ = len(self.classes_) + + # Meta-training loop + for epoch in range(self.meta_epochs): + meta_loss = 0 + + for _ in range(self.num_tasks): + # Create a task + (support_X, support_y), (query_X, query_y) = self._create_task(X, y, self.n_classes_) + + # Compute prototypes + prototypes = self._compute_prototypes(support_X, support_y) + + # Compute distances to prototypes for query set + query_embeddings = self.embedding_net.predict(query_X) + + distances = [] + for query_emb in query_embeddings: + dists = [self._euclidean_distance(query_emb.reshape(1, -1), proto.reshape(1, -1))[0] + for proto in prototypes] + distances.append(dists) + + distances = np.array(distances) + + # Predict based on nearest prototype + predictions = np.argmin(distances, axis=1) + + # Compute loss + task_loss = 1 - accuracy_score(query_y, predictions) + meta_loss += task_loss + + meta_loss /= self.num_tasks + + if epoch % 10 == 0: + logger.info(f"Meta-epoch {epoch}/{self.meta_epochs}, Meta-loss: {meta_loss:.4f}") + + self.is_fitted = True + logger.info("Prototypical Network meta-training completed.") + return self + + def predict(self, X): + """Predict using the prototypical network.""" + if not self.is_fitted: + raise ValueError("Model must be fitted before making predictions.") + + # For prediction, we need support examples to compute prototypes + # This is a simplified version + embeddings = self.embedding_net.predict(X) + # Return dummy predictions - in practice, you'd need support examples + return np.zeros(len(X), dtype=int) + + +class DomainAdaptation: + """ + Domain adaptation utilities for transfer learning. + + Provides methods to adapt models trained on source domain to target domain. + """ + + def __init__(self, base_model=None): + """ + Initialize domain adaptation utilities. + + Args: + base_model: Base model to adapt + """ + self.base_model = base_model + self.is_adapted = False + + def adapt_model(self, source_X, source_y, target_X, target_y=None, + adaptation_method='fine_tune', **kwargs): + """ + Adapt a model from source domain to target domain. + + Args: + source_X: Source domain features + source_y: Source domain labels + target_X: Target domain features + target_y: Target domain labels (if available) + adaptation_method: Method to use ('fine_tune', 'domain_adversarial', 'maml') + **kwargs: Additional arguments for adaptation + + Returns: + Adapted model + """ + if adaptation_method == 'fine_tune': + return self._fine_tune_adaptation(source_X, source_y, target_X, target_y, **kwargs) + elif adaptation_method == 'domain_adversarial': + return self._domain_adversarial_adaptation(source_X, source_y, target_X, **kwargs) + elif adaptation_method == 'maml': + return self._maml_adaptation(source_X, source_y, target_X, target_y, **kwargs) + else: + raise ValueError(f"Unknown adaptation method: {adaptation_method}") + + def _fine_tune_adaptation(self, source_X, source_y, target_X, target_y, + learning_rate=0.001, epochs=10): + """Fine-tune the model on target domain.""" + if self.base_model is None: + raise ValueError("Base model must be provided for fine-tuning.") + + # Train on source domain first + self.base_model.fit(source_X, source_y) + + # Fine-tune on target domain if labels are available + if target_y is not None: + # Use a smaller learning rate for fine-tuning + if hasattr(self.base_model, 'learning_rate_init'): + original_lr = self.base_model.learning_rate_init + self.base_model.learning_rate_init = learning_rate + + self.base_model.fit(target_X, target_y) + + if hasattr(self.base_model, 'learning_rate_init'): + self.base_model.learning_rate_init = original_lr + + self.is_adapted = True + return self.base_model + + def _domain_adversarial_adaptation(self, source_X, source_y, target_X, + lambda_weight=0.1): + """Domain adversarial training for unsupervised domain adaptation.""" + # This is a simplified implementation + # In practice, you'd implement a domain classifier and adversarial training + logger.warning("Domain adversarial adaptation is not fully implemented.") + return self.base_model + + def _maml_adaptation(self, source_X, source_y, target_X, target_y, + inner_steps=5, inner_lr=0.01): + """Use MAML for domain adaptation.""" + maml = MAMLClassifier(inner_steps=inner_steps, inner_lr=inner_lr) + + # Combine source and target data for meta-training + combined_X = np.vstack([source_X, target_X]) + combined_y = np.hstack([source_y, target_y]) + + maml.fit(combined_X, combined_y) + return maml + + +class TransferLearning: + """ + Transfer learning utilities for leveraging pre-trained models. + """ + + def __init__(self, base_model=None): + """ + Initialize transfer learning utilities. + + Args: + base_model: Pre-trained base model + """ + self.base_model = base_model + + def transfer_features(self, source_model, target_X, feature_layer='penultimate'): + """ + Extract features from a pre-trained model for transfer learning. + + Args: + source_model: Pre-trained source model + target_X: Target data + feature_layer: Which layer to extract features from + + Returns: + Extracted features + """ + if hasattr(source_model, 'predict_proba'): + # For sklearn models, use predict_proba as feature extractor + features = source_model.predict_proba(target_X) + else: + # For other models, use predict as fallback + features = source_model.predict(target_X) + + return features + + def create_transfer_model(self, source_model, target_X, target_y, + transfer_method='feature_extraction'): + """ + Create a transfer learning model. + + Args: + source_model: Pre-trained source model + target_X: Target data + target_y: Target labels + transfer_method: Method to use ('feature_extraction', 'fine_tuning') + + Returns: + Transfer learning model + """ + if transfer_method == 'feature_extraction': + # Extract features and train a new classifier + features = self.transfer_features(source_model, target_X) + + from sklearn.linear_model import LogisticRegression + transfer_model = LogisticRegression(random_state=42) + transfer_model.fit(features, target_y) + + return transfer_model + + elif transfer_method == 'fine_tuning': + # Fine-tune the entire model + if hasattr(source_model, 'fit'): + source_model.fit(target_X, target_y) + return source_model + else: + raise ValueError("Source model must have a fit method for fine-tuning.") + + else: + raise ValueError(f"Unknown transfer method: {transfer_method}") + + +# Utility functions for few-shot learning +def create_few_shot_dataset(X, y, n_way=2, k_shot=5, n_query=5, random_state=42): + """ + Create a few-shot learning dataset. + + Args: + X: Features + y: Labels + n_way: Number of classes per task + k_shot: Number of examples per class for support set + n_query: Number of examples per class for query set + random_state: Random seed + + Returns: + List of tasks, each containing (support_X, support_y, query_X, query_y) + """ + np.random.seed(random_state) + tasks = [] + + unique_classes = np.unique(y) + if len(unique_classes) < n_way: + raise ValueError(f"Not enough classes. Need at least {n_way}, got {len(unique_classes)}") + + # Create multiple tasks + for _ in range(10): # Create 10 tasks + selected_classes = np.random.choice(unique_classes, n_way, replace=False) + + support_X, support_y = [], [] + query_X, query_y = [], [] + + for i, class_label in enumerate(selected_classes): + class_indices = np.where(y == class_label)[0] + if len(class_indices) >= k_shot + n_query: + selected_indices = np.random.choice(class_indices, k_shot + n_query, replace=False) + + support_indices = selected_indices[:k_shot] + query_indices = selected_indices[k_shot:] + + support_X.extend(X[support_indices]) + support_y.extend([i] * k_shot) + + query_X.extend(X[query_indices]) + query_y.extend([i] * n_query) + + if len(support_X) == n_way * k_shot and len(query_X) == n_way * n_query: + tasks.append(( + np.array(support_X), np.array(support_y), + np.array(query_X), np.array(query_y) + )) + + return tasks + + +def evaluate_few_shot_model(model, tasks): + """ + Evaluate a few-shot learning model on multiple tasks. + + Args: + model: Few-shot learning model + tasks: List of tasks to evaluate on + + Returns: + Dictionary with evaluation metrics + """ + accuracies = [] + + for support_X, support_y, query_X, query_y in tasks: + # Adapt model to the task + if hasattr(model, '_adapt_to_task'): + adapted_model = model._adapt_to_task(support_X, support_y) + else: + # For models that don't have explicit adaptation + adapted_model = model + + # Predict on query set + predictions = adapted_model.predict(query_X) + accuracy = accuracy_score(query_y, predictions) + accuracies.append(accuracy) + + return { + 'mean_accuracy': np.mean(accuracies), + 'std_accuracy': np.std(accuracies), + 'accuracies': accuracies + } \ No newline at end of file diff --git a/igel/igel/gpu_utils.py b/igel/igel/gpu_utils.py new file mode 100644 index 0000000..ac25a72 --- /dev/null +++ b/igel/igel/gpu_utils.py @@ -0,0 +1,34 @@ +def detect_gpu(): + """ + Detect if a GPU is available for PyTorch or TensorFlow. + Returns a string describing the GPU status. + """ + try: + import torch + if torch.cuda.is_available(): + return f"PyTorch GPU available: {torch.cuda.get_device_name(0)}" + except ImportError: + pass + try: + import tensorflow as tf + gpus = tf.config.list_physical_devices('GPU') + if gpus: + return f"TensorFlow GPU available: {gpus[0].name}" + except ImportError: + pass + return "No GPU detected" + + +def report_gpu_utilization(): + """ + Report GPU utilization using GPUtil if available. + """ + try: + import GPUtil + gpus = GPUtil.getGPUs() + for gpu in gpus: + print(f"GPU {gpu.id}: {gpu.name}, load: {gpu.load*100:.1f}%, memory: {gpu.memoryUsed}/{gpu.memoryTotal}MB") + except ImportError: + print("GPUtil not installed. Skipping detailed GPU utilization.") + except Exception as e: + print(f"Error reporting GPU utilization: {e}") \ No newline at end of file diff --git a/igel/igel/hyperparams.py b/igel/igel/hyperparams.py new file mode 100644 index 0000000..ba0c79f --- /dev/null +++ b/igel/igel/hyperparams.py @@ -0,0 +1,25 @@ +from sklearn.model_selection import GridSearchCV, RandomizedSearchCV + + +def hyperparameter_search(model, + method, + params, + x_train, + y_train, + **kwargs): + + search = None + if method == 'grid_search': + search = GridSearchCV(model, + params, + **kwargs) + + elif method == 'random_search': + search = RandomizedSearchCV(model, + params, + **kwargs) + else: + raise Exception("hyperparameter method must be grid_search or random_search") + + search.fit(x_train, y_train) + return search.best_estimator_, search.best_score_, search.best_params_ diff --git a/igel/igel/igel.py b/igel/igel/igel.py new file mode 100644 index 0000000..d75fece --- /dev/null +++ b/igel/igel/igel.py @@ -0,0 +1,862 @@ +"""Main module.""" + +import json +import logging +import os +import warnings +import datetime + +import joblib +import numpy as np +import pandas as pd + +try: + from igel.configs import configs + from igel.data import evaluate_model, metrics_dict, models_dict + from igel.hyperparams import hyperparameter_search + from igel.preprocessing import ( + encode, + handle_missing_values, + normalize, + read_data_to_df, + update_dataset_props, + ) + from igel.utils import ( + _reshape, + create_yaml, + extract_params, + read_json, + read_yaml, + ) + from igel.model_registry import ModelRegistry +except ImportError: + from igel.utils import ( + read_yaml, + create_yaml, + extract_params, + _reshape, + read_json, + ) + from data import evaluate_model + from configs import configs + from data import models_dict, metrics_dict + from preprocessing import update_dataset_props + from preprocessing import ( + handle_missing_values, + encode, + normalize, + read_data_to_df, + ) + from hyperparams import hyperparameter_search + from model_registry import ModelRegistry + +from sklearn.model_selection import cross_validate, train_test_split +from sklearn.multioutput import MultiOutputClassifier, MultiOutputRegressor +from sklearn.pipeline import Pipeline +from sklearn.preprocessing import StandardScaler +from sklearn.ensemble import RandomForestClassifier + +from skl2onnx import convert_sklearn +from skl2onnx.common.data_types import FloatTensorType + +warnings.filterwarnings("ignore") +logging.basicConfig(format="%(levelname)s - %(message)s", level=logging.INFO) +logger = logging.getLogger(__name__) + + +class Igel: + """ + Igel is the base model to use the fit, evaluate and predict functions of the sklearn library + """ + + available_commands = ("fit", "evaluate", "predict", "experiment","export") + supported_types = ("regression", "classification", "clustering") + results_path = configs.get("results_path") # path to the results folder + default_model_path = configs.get( + "default_model_path" + ) # path to the pre-fitted model + default_onnx_model_path = configs.get( + "default_onnx_model_path" + ) # path to the onnx-model + description_file = configs.get( + "description_file" + ) # path to the description.json file + evaluation_file = configs.get( + "evaluation_file" + ) # path to the evaluation.json file + prediction_file = configs.get( + "prediction_file" + ) # path to the predictions.csv + default_dataset_props = configs.get( + "dataset_props" + ) # dataset props that can be changed from the yaml file + default_model_props = configs.get( + "model_props" + ) # model props that can be changed from the yaml file + model = None + predictions = None # store predictions as pandas df + + def __init__(self, **cli_args): + logger.info(f"Entered CLI args: {cli_args}") + logger.info(f"Executing command: {cli_args.get('cmd')} ...") + self.data_path: str = str( + cli_args.get("data_path") + ) # path to the dataset + logger.info(f"reading data from {self.data_path}") + + self.command = cli_args.get("cmd", None) + if not self.command or self.command not in self.available_commands: + raise Exception( + f"You must enter a valid command.\n" + f"available commands: {self.available_commands}" + ) + + if self.command == "fit": + self.yml_path = str(cli_args.get("yaml_path")) + file_ext = self.yml_path.split(".")[-1] + logger.info(f"You passed the configurations as a {file_ext} file.") + + self.yaml_configs = ( + read_yaml(self.yml_path) + if file_ext == "yaml" + else read_json(self.yml_path) + ) + logger.info(f"your chosen configuration: {self.yaml_configs}") + + # dataset options given by the user + self.dataset_props: dict = self.yaml_configs.get( + "dataset", self.default_dataset_props + ) + # model options given by the user + self.model_props: dict = self.yaml_configs.get( + "model", self.default_model_props + ) + # list of target(s) to predict + self.target: list = self.yaml_configs.get("target") + + self.model_type: str = self.model_props.get("type") + logger.info( + f"dataset_props: {self.dataset_props} \n" + f"model_props: {self.model_props} \n " + f"target: {self.target} \n" + ) + + # handle random numbers generation + random_num_options = self.dataset_props.get("random_numbers", None) + if random_num_options: + generate_reproducible = random_num_options.get( + "generate_reproducible", None + ) + if generate_reproducible: + logger.info( + "You provided the generate reproducible results option." + ) + seed = random_num_options.get("seed", 42) + np.random.seed(seed) + logger.info( + f"Setting a seed = {seed} to generate same random numbers on each experiment.." + ) + + # if entered command is export, then the pre-fitted model needs to be loaded and converted to onnx + elif self.command == "export": + self.model_path = cli_args.get( + "model_path", self.default_model_path + ) + logger.info(f"path of the pre-fitted model => {self.model_path}") + + # if entered command is evaluate or predict, then the pre-fitted model needs to be loaded and used + else: + self.model_path = cli_args.get( + "model_path", self.default_model_path + ) + logger.info(f"path of the pre-fitted model => {self.model_path}") + + self.prediction_file = cli_args.get( + "prediction_file", self.prediction_file + ) + + # set description.json if provided: + self.description_file = cli_args.get( + "description_file", self.description_file + ) + + # load description file to read stored training parameters + with open(self.description_file) as f: + dic = json.load(f) + self.target: list = dic.get( + "target" + ) # target to predict as a list + self.model_type: str = dic.get( + "type" + ) # type of the model -> regression, classification or clustering + self.dataset_props: dict = dic.get( + "dataset_props" + ) # dataset props entered while fitting + getattr(self, self.command)() + + def _create_model(self, **kwargs): + """ + fetch a model depending on the provided type and algorithm by the user and return it + If a custom_model_class is provided (as a class or instance), use it instead. + @return: class of the chosen model + """ + custom_model_class = kwargs.pop("custom_model_class", None) + if custom_model_class is not None: + # If it's a class, instantiate it; if it's already an instance, use as is + if isinstance(custom_model_class, type): + model = custom_model_class() + else: + model = custom_model_class + model_args = "custom_model" + return model, model_args + model_type: str = self.model_props.get("type") + model_algorithm: str = self.model_props.get("algorithm") + use_cv = self.model_props.get("use_cv_estimator", None) + + model_args = None + if not model_type or not model_algorithm: + raise Exception(f"model_type and algorithm cannot be None") + algorithms: dict = models_dict.get( + model_type + ) # extract all algorithms as a dictionary + model = algorithms.get( + model_algorithm + ) # extract model class depending on the algorithm + logger.info( + f"Solving a {model_type} problem using ===> {model_algorithm}" + ) + if not model: + raise Exception("Model not found in the algorithms list") + else: + model_props_args = self.model_props.get("arguments", None) + if model_props_args and type(model_props_args) == dict: + model_args = model_props_args + elif not model_props_args or model_props_args.lower() == "default": + model_args = None + + if use_cv: + model_class = model.get("cv_class", None) + if model_class: + logger.info( + f"cross validation estimator detected. " + f"Switch to the CV version of the {model_algorithm} algorithm" + ) + else: + logger.info( + f"No CV class found for the {model_algorithm} algorithm" + ) + else: + model_class = model.get("class") + logger.info( + f"model arguments: \n" f"{self.model_props.get('arguments')}" + ) + model = ( + model_class(**kwargs) + if not model_args + else model_class(**model_args) + ) + return model, model_args + + def _save_model(self, model): + """ + save the fitted model to disk + @param model: fitted model to save + """ + # Save model to default location + joblib.dump(model, self.default_model_path) + + # Register model in the registry + registry = ModelRegistry() + + # Prepare metadata + metadata = { + "model_type": self.model_type, + "model_algorithm": self.model_props.get("algorithm"), + "target": self.target, + "dataset_props": self.dataset_props, + "model_props": self.model_props, + "timestamp": datetime.datetime.now().isoformat() + } + + # Register the model + model_id = registry.register_model( + model_path=self.default_model_path, + model_name=f"{self.model_type}_{self.model_props.get('algorithm')}", + version="1.0", # This could be made configurable + metadata=metadata + ) + + logger.info(f"Model saved and registered with ID: {model_id}") + + # Save model description + description = { + "type": self.model_type, + "target": self.target, + "dataset_props": self.dataset_props, + "model_props": self.model_props, + "model_id": model_id + } + + with open(self.description_file, "w") as f: + json.dump(description, f, indent=2) + + def _load_model(self, f: str = ""): + """ + load a saved model from file + @param f: path to model + @return: loaded model + """ + try: + if not f: + logger.info(f"result path: {self.results_path} ") + logger.info(f"loading model form {self.default_model_path} ") + model = joblib.load(open(self.default_model_path, "rb")) + else: + logger.info(f"loading from {f}") + model = joblib.load(open(f, "rb")) + return model + except FileNotFoundError: + logger.error(f"File not found in {self.default_model_path} ") + + def _prepare_fit_data(self): + return self._process_data(target="fit") + + def _prepare_eval_data(self): + return self._process_data(target="evaluate") + + def _process_data(self, target="fit"): + """ + read and return data as x and y + @return: list of separate x and y + """ + + if self.model_type != "clustering": + assert isinstance( + self.target, list + ), "provide target(s) as a list in the yaml file" + assert ( + len(self.target) > 0 + ), "please provide at least a target to predict" + + try: + read_data_options = self.dataset_props.get("read_data_options", {}) + dataset = read_data_to_df( + data_path=self.data_path, **read_data_options + ) + logger.info(f"dataset shape: {dataset.shape}") + attributes = list(dataset.columns) + logger.info(f"dataset attributes: {attributes}") + + # handle missing values in the dataset + preprocess_props = self.dataset_props.get("preprocess", None) + if preprocess_props: + # handle encoding + encoding = preprocess_props.get("encoding") + if encoding: + encoding_type = encoding.get("type", None) + column = encoding.get("column", None) + if column in attributes: + dataset, classes_map = encode( + df=dataset, + encoding_type=encoding_type.lower(), + column=column, + ) + if classes_map: + self.dataset_props[ + "label_encoding_classes" + ] = classes_map + logger.info( + f"adding classes_map to dataset props: \n{classes_map}" + ) + logger.info( + f"shape of the dataset after encoding => {dataset.shape}" + ) + + # preprocessing strategy: mean, median, mode etc.. + strategy = preprocess_props.get("missing_values") + if strategy: + dataset = handle_missing_values(dataset, strategy=strategy) + logger.info( + f"shape of the dataset after handling missing values => {dataset.shape}" + ) + + if target == "predict" or target == "fit_cluster": + x = _reshape(dataset.to_numpy()) + if not preprocess_props: + return x + scaling_props = preprocess_props.get("scale", None) + if not scaling_props: + return x + else: + scaling_method = scaling_props.get("method", None) + return normalize(x, method=scaling_method) + + if any(col not in attributes for col in self.target): + raise Exception( + "chosen target(s) to predict must exist in the dataset" + ) + + y = pd.concat([dataset.pop(x) for x in self.target], axis=1) + x = _reshape(dataset.to_numpy()) + y = _reshape(y.to_numpy()) + logger.info(f"y shape: {y.shape} and x shape: {x.shape}") + + # handle data scaling + if preprocess_props: + scaling_props = preprocess_props.get("scale", None) + if scaling_props: + scaling_method = scaling_props.get("method", None) + scaling_target = scaling_props.get("target", None) + if scaling_target == "all": + x = normalize(x, method=scaling_method) + y = normalize(y, method=scaling_method) + elif scaling_target == "inputs": + x = normalize(x, method=scaling_method) + elif scaling_target == "outputs": + y = normalize(y, method=scaling_method) + + if target == "evaluate": + return x, y + + split_options = self.dataset_props.get("split", None) + if not split_options: + return x, y, None, None + test_size = split_options.get("test_size") + shuffle = split_options.get("shuffle") + stratify = split_options.get("stratify") + x_train, x_test, y_train, y_test = train_test_split( + x, + y, + test_size=test_size, + shuffle=shuffle, + stratify=None + if not stratify or stratify.lower() == "default" + else stratify, + ) + + return x_train, y_train, x_test, y_test + + except Exception as e: + logger.exception(f"error occured while preparing the data: {e}") + + def _prepare_clustering_data(self): + """ + preprocess data for the clustering algorithm + """ + return self._process_data(target="fit_cluster") + + def _prepare_predict_data(self): + """ + preprocess predict data to get similar data to the one used when training the model + """ + return self._process_data(target="predict") + + def get_evaluation(self, model, x_test, y_true, y_pred, **kwargs): + try: + res = evaluate_model( + model_type=self.model_type, + model=model, + x_test=x_test, + y_pred=y_pred, + y_true=y_true, + get_score_only=False, + **kwargs, + ) + except Exception as e: + logger.debug(e) + res = evaluate_model( + model_type=self.model_type, + model=model, + x_test=x_test, + y_pred=y_pred, + y_true=y_true, + get_score_only=True, + **kwargs, + ) + return res + + def fit(self, **kwargs): + """ + fit a machine learning model and save it to a file along with a description.json file + @return: None + """ + x_train = None + x_test = None + y_train = None + y_test = None + cv_results = None + eval_results = None + cv_params = None + hp_search_results = {} + + if self.model_type == "clustering": + x_train = self._prepare_clustering_data() + else: + x_train, y_train, x_test, y_test = self._prepare_fit_data() + self.model, model_args = self._create_model(**kwargs) + logger.info(f"executing a {self.model.__class__.__name__} algorithm...") + + # convert to multioutput if there is more than one target to predict: + if self.model_type != "clustering" and len(self.target) > 1: + logger.info( + f"predicting multiple targets detected. Hence, the model will be automatically " + f"converted to a multioutput model" + ) + self.model = ( + MultiOutputClassifier(self.model) + if self.model_type == "classification" + else MultiOutputRegressor(self.model) + ) + + if self.model_type != "clustering": + cv_params = self.model_props.get("cross_validate", None) + if not cv_params: + logger.info(f"cross validation is not provided") + else: + # perform cross validation + logger.info("performing cross validation ...") + cv_results = cross_validate( + estimator=self.model, X=x_train, y=y_train, **cv_params + ) + hyperparams_props = self.model_props.get( + "hyperparameter_search", None + ) + if hyperparams_props: + + # perform hyperparameter search + method = hyperparams_props.get("method", None) + grid_params = hyperparams_props.get("parameter_grid", None) + hp_args = hyperparams_props.get("arguments", None) + logger.info( + f"Performing hyperparameter search using -> {method}" + ) + logger.info( + f"Grid parameters entered by the user: {grid_params}" + ) + logger.info(f"Additional hyperparameter arguments: {hp_args}") + best_estimator, best_params, best_score = hyperparameter_search( + model=self.model, + method=method, + params=grid_params, + x_train=x_train, + y_train=y_train, + **hp_args, + ) + hp_search_results["best_params"] = best_params + hp_search_results["best_score"] = best_score + self.model = best_estimator + + self.model.fit(x_train, y_train) + + else: # if the model type is clustering + self.model.fit(x_train) + + saved = self._save_model(self.model) + if saved: + logger.info( + f"model saved successfully and can be found in the {self.results_path} folder" + ) + + if self.model_type == "clustering": + eval_results = self.model.score(x_train) + else: + if x_test is None: + logger.info( + f"no split options was provided. training score will be calculated" + ) + eval_results = self.model.score(x_train, y_train) + + else: + logger.info( + f"split option detected. The performance will be automatically evaluated " + f"using the test data portion" + ) + y_pred = self.model.predict(x_test) + eval_results = self.get_evaluation( + model=self.model, + x_test=x_test, + y_true=y_test, + y_pred=y_pred, + **kwargs, + ) + + fit_description = { + "model": self.model.__class__.__name__, + "arguments": model_args if model_args else "default", + "type": self.model_props["type"], + "algorithm": self.model_props["algorithm"], + "dataset_props": self.dataset_props, + "model_props": self.model_props, + "data_path": self.data_path, + "train_data_shape": x_train.shape, + "test_data_shape": None if x_test is None else x_test.shape, + "train_data_size": x_train.shape[0], + "test_data_size": None if x_test is None else x_test.shape[0], + "results_path": str(self.results_path), + "model_path": str(self.default_model_path), + "target": None if self.model_type == "clustering" else self.target, + "results_on_test_data": eval_results, + "hyperparameter_search_results": hp_search_results, + } + if self.model_type == "clustering": + clustering_res = { + "cluster_centers": self.model.cluster_centers_.tolist(), + "cluster_labels": self.model.labels_.tolist(), + } + fit_description["clustering_results"] = clustering_res + + if cv_params: + cv_res = { + "fit_time": cv_results["fit_time"].tolist(), + "score_time": cv_results["score_time"].tolist(), + "test_score": cv_results["test_score"].tolist(), + } + fit_description["cross_validation_params"] = cv_params + fit_description["cross_validation_results"] = cv_res + + try: + logger.info(f"saving fit description to {self.description_file}") + with open(self.description_file, "w", encoding="utf-8") as f: + json.dump(fit_description, f, ensure_ascii=False, indent=4) + except Exception as e: + logger.exception( + f"Error while storing the fit description file: {e}" + ) + + def evaluate(self, **kwargs): + """ + evaluate a pre-fitted model and save results to a evaluation.json + @return: None + """ + x_val = None + y_true = None + eval_results = None + + try: + model = self._load_model() + if self.model_type != "clustering": + x_val, y_true = self._prepare_eval_data() + y_pred = model.predict(x_val) + eval_results = self.get_evaluation( + model=model, + x_test=x_val, + y_true=y_true, + y_pred=y_pred, + **kwargs, + ) + else: + x_val = self._prepare_clustering_data() + y_pred = model.predict(x_val) + eval_results = model.score(x_val, y_pred) + + logger.info(f"saving fit description to {self.evaluation_file}") + with open(self.evaluation_file, "w", encoding="utf-8") as f: + json.dump(eval_results, f, ensure_ascii=False, indent=4) + + except Exception as e: + logger.exception(f"error occured during evaluation: {e}") + + def _get_predictions(self, **kwargs): + """ + use a pre-fitted model to generate predictions + @return: None + """ + try: + model = self._load_model(f=self.model_path) + x_val = ( + self._prepare_predict_data() + ) # the same is used for clustering + y_pred = model.predict(x_val) + y_pred = _reshape(y_pred) + logger.info( + f"predictions shape: {y_pred.shape} | shape len: {len(y_pred.shape)}" + ) + logger.info(f"predict on targets: {self.target}") + if not self.target: + self.target = ["result"] + df_pred = pd.DataFrame.from_dict( + { + self.target[i]: y_pred[:, i] + if len(y_pred.shape) > 1 + else y_pred + for i in range(len(self.target)) + } + ) + return df_pred + + except Exception as e: + logger.exception(f"Error while preparing predictions: {e}") + + def predict(self): + """ + generate predictions and save them as csv. This is used as a command from cli + """ + + df_pred = self._get_predictions() + self.predictions = df_pred + logger.info(f"saving the predictions to {self.prediction_file}") + df_pred.to_csv(self.prediction_file, index=False) + + def export(self): + """ + export a sklearn model to ONNX. This is used as a command from cli + @return: None + """ + try: + + logger.info( + f"Trying to load sklearn model from directory - {self.model_path} " + ) + model = self._load_model(f=self.model_path) + initial_type = [('float_input', FloatTensorType([None, 4]))] + onx = convert_sklearn(model, initial_types=initial_type) + + # check if model_results folder is present and create if absent + if not os.path.exists(self.results_path): + logger.info( + f"creating model_results folder to save results...\n" + f"path of the results folder: {self.results_path}" + ) + os.mkdir(self.results_path) + else: + logger.info(f"Folder {self.results_path} already exists") + logger.warning( + f"data in the {self.results_path} folder will be overridden. If you don't " + f"want this, then move the current {self.results_path} to another path" + ) + + with open(self.default_onnx_model_path, "wb") as f: + f.write(onx.SerializeToString()) + logger.info( + f"Successfully saved exported onnx model at - {self.default_onnx_model_path} " + ) + except Exception as e: + logger.exception(f"Error while exporting model: {e}") + + @staticmethod + def create_init_mock_file( + model_type=None, model_name=None, target=None, *args, **kwargs + ): + path = configs.get("init_file_path", None) + if not path: + raise Exception("You need to provide a path for the init file") + + dataset_props = Igel.default_dataset_props + model_props = Igel.default_model_props + if model_type: + logger.info(f"user selected model type = {model_type}") + model_props["type"] = model_type + if model_name: + logger.info(f"user selected algorithm = {model_name}") + model_props["algorithm"] = model_name + + logger.info(f"initalizing a default igel.yaml in {path}") + default_data = { + "dataset": dataset_props, + "model": model_props, + "target": ["provide your target(s) here"] + if not target + else [tg for tg in target.split()], + } + created = create_yaml(default_data, path) + if created: + logger.info( + f"a default igel.yaml is created for you in {path}. " + f"you just need to overwrite the values to meet your expectations" + ) + else: + logger.warning( + f"something went wrong while initializing a default file" + ) + + @staticmethod + def list_models(model_name: str = None): + """ + List all registered models or filter by model name. + + Args: + model_name (str, optional): Filter models by name + + Returns: + List[Dict]: List of model metadata + """ + registry = ModelRegistry() + return registry.list_models(model_name) + + @staticmethod + def get_model_info(model_id: str): + """ + Get detailed information about a specific model. + + Args: + model_id (str): Model ID to retrieve + + Returns: + Dict: Model metadata + """ + registry = ModelRegistry() + return registry.get_model(model_id) + + @staticmethod + def delete_model(model_id: str): + """ + Delete a model from the registry. + + Args: + model_id (str): Model ID to delete + + Returns: + bool: True if successful, False otherwise + """ + registry = ModelRegistry() + return registry.delete_model(model_id) + + @staticmethod + def update_model_metadata(model_id: str, metadata: dict): + """ + Update metadata for a registered model. + + Args: + model_id (str): Model ID to update + metadata (dict): New metadata to add/update + + Returns: + bool: True if successful, False otherwise + """ + registry = ModelRegistry() + return registry.update_metadata(model_id, metadata) + +def validate_data(df: pd.DataFrame, target: str = None) -> None: + """ + Validates the input DataFrame for missing values, invalid types, and out-of-range values. + Raises ValueError with a clear message if validation fails. + """ + # Check for missing values + if df.isnull().values.any(): + raise ValueError("Input data contains missing values. Please clean your data before training.") + + # Check for invalid types (non-numeric in features, if required) + for col in df.columns: + if col != target and not pd.api.types.is_numeric_dtype(df[col]): + raise ValueError(f"Column '{col}' contains non-numeric data. Please ensure all features are numeric.") + + # (Optional) Check for out-of-range values (example: negative values in features that should be positive) + # for col in df.columns: + # if (df[col] < 0).any(): + # raise ValueError(f"Column '{col}' contains negative values, which are not allowed.") + + # Add more checks as needed + +def build_pipeline(config: dict): + steps = [] + for step in config.get("pipeline", []): + if step["type"] == "scaler": + if step["algorithm"] == "StandardScaler": + steps.append(("scaler", StandardScaler())) + # Add more scalers as needed + elif step["type"] == "model": + if step["algorithm"] == "RandomForestClassifier": + params = step.get("params", {}) + steps.append(("model", RandomForestClassifier(**params))) + # Add more models as needed + return Pipeline(steps) diff --git a/igel/igel/leaderboard.py b/igel/igel/leaderboard.py new file mode 100644 index 0000000..7111024 --- /dev/null +++ b/igel/igel/leaderboard.py @@ -0,0 +1,21 @@ +import os +import pandas as pd + +def generate_leaderboard(results_dir="results", output_csv="leaderboard.csv", output_html="leaderboard.html"): + rows = [] + for fname in os.listdir(results_dir): + if fname.endswith(".json"): + data = pd.read_json(os.path.join(results_dir, fname), typ='series') + rows.append(data) + elif fname.endswith(".csv"): + data = pd.read_csv(os.path.join(results_dir, fname)).iloc[0] + rows.append(data) + if not rows: + print("No result files found.") + return + df = pd.DataFrame(rows) + df = df.sort_values(by=df.columns[1], ascending=False) # Sort by first metric column + df.to_csv(output_csv, index=False) + df.to_html(output_html, index=False) + print(f"Leaderboard saved to {output_csv} and {output_html}") + print(df) diff --git a/igel/igel/livestock_management.py b/igel/igel/livestock_management.py new file mode 100644 index 0000000..513091d --- /dev/null +++ b/igel/igel/livestock_management.py @@ -0,0 +1,19 @@ +""" +Livestock Management Utilities + +- Animal health monitoring (placeholder) +- Breeding optimization (placeholder) +- Feed optimization (placeholder) +""" + +def monitor_animal_health(animal_data): + # Placeholder: Add real health monitoring logic here + return {"status": "healthy", "recommendation": "no action needed"} + +def optimize_breeding(herd_data): + # Placeholder: Add real breeding optimization logic here + return {"breeding_plan": "optimal", "expected_gain": 5} + +def optimize_feed(feed_data): + # Placeholder: Add real feed optimization logic here + return {"feed_mix": "balanced", "cost": 100} diff --git a/igel/igel/model_registry.py b/igel/igel/model_registry.py new file mode 100644 index 0000000..2a304a6 --- /dev/null +++ b/igel/igel/model_registry.py @@ -0,0 +1,206 @@ +""" +Model Registry System for Igel. + +This module provides functionality for storing, versioning, and managing trained models. +""" + +import os +import json +import datetime +from pathlib import Path +from typing import Dict, List, Optional, Any +import shutil +import hashlib + +class ModelRegistry: + """A class to manage model versions and metadata.""" + + def __init__(self, registry_path: str = "model_registry"): + """ + Initialize the model registry. + + Args: + registry_path (str): Path to store the model registry + """ + self.registry_path = Path(registry_path) + self.metadata_path = self.registry_path / "metadata" + self.models_path = self.registry_path / "models" + + # Create necessary directories + self.registry_path.mkdir(exist_ok=True) + self.metadata_path.mkdir(exist_ok=True) + self.models_path.mkdir(exist_ok=True) + + # Initialize metadata index + self.metadata_index_path = self.metadata_path / "index.json" + if not self.metadata_index_path.exists(): + self._save_metadata_index({}) + + def _save_metadata_index(self, index: Dict[str, Any]) -> None: + """Save the metadata index to disk.""" + with open(self.metadata_index_path, 'w') as f: + json.dump(index, f, indent=2) + + def _load_metadata_index(self) -> Dict[str, Any]: + """Load the metadata index from disk.""" + if not self.metadata_index_path.exists(): + return {} + with open(self.metadata_index_path, 'r') as f: + return json.load(f) + + def register_model(self, + model_path: str, + model_name: str, + version: str, + metadata: Dict[str, Any]) -> str: + """ + Register a new model version. + + Args: + model_path (str): Path to the model file + model_name (str): Name of the model + version (str): Version identifier + metadata (Dict[str, Any]): Additional metadata about the model + + Returns: + str: Model ID + """ + # Generate unique model ID + timestamp = datetime.datetime.now().isoformat() + model_id = f"{model_name}_{version}_{hashlib.md5(timestamp.encode()).hexdigest()[:8]}" + + # Create model directory + model_dir = self.models_path / model_id + model_dir.mkdir(exist_ok=True) + + # Copy model file + shutil.copy2(model_path, model_dir / "model.pkl") + + # Prepare metadata + model_metadata = { + "model_id": model_id, + "model_name": model_name, + "version": version, + "timestamp": timestamp, + "path": str(model_dir), + **metadata + } + + # Save metadata + metadata_file = self.metadata_path / f"{model_id}.json" + with open(metadata_file, 'w') as f: + json.dump(model_metadata, f, indent=2) + + # Update index + index = self._load_metadata_index() + if model_name not in index: + index[model_name] = [] + index[model_name].append(model_id) + self._save_metadata_index(index) + + return model_id + + def get_model(self, model_id: str) -> Optional[Dict[str, Any]]: + """ + Retrieve model information by ID. + + Args: + model_id (str): Model ID to retrieve + + Returns: + Optional[Dict[str, Any]]: Model metadata if found, None otherwise + """ + metadata_file = self.metadata_path / f"{model_id}.json" + if not metadata_file.exists(): + return None + + with open(metadata_file, 'r') as f: + return json.load(f) + + def list_models(self, model_name: Optional[str] = None) -> List[Dict[str, Any]]: + """ + List all registered models or models with a specific name. + + Args: + model_name (Optional[str]): Filter by model name + + Returns: + List[Dict[str, Any]]: List of model metadata + """ + index = self._load_metadata_index() + models = [] + + if model_name: + if model_name not in index: + return [] + model_ids = index[model_name] + else: + model_ids = [mid for ids in index.values() for mid in ids] + + for model_id in model_ids: + model_info = self.get_model(model_id) + if model_info: + models.append(model_info) + + return models + + def delete_model(self, model_id: str) -> bool: + """ + Delete a model and its metadata. + + Args: + model_id (str): Model ID to delete + + Returns: + bool: True if successful, False otherwise + """ + model_info = self.get_model(model_id) + if not model_info: + return False + + # Remove model files + model_dir = Path(model_info["path"]) + if model_dir.exists(): + shutil.rmtree(model_dir) + + # Remove metadata + metadata_file = self.metadata_path / f"{model_id}.json" + if metadata_file.exists(): + metadata_file.unlink() + + # Update index + index = self._load_metadata_index() + for model_name, model_ids in index.items(): + if model_id in model_ids: + index[model_name].remove(model_id) + if not index[model_name]: + del index[model_name] + break + self._save_metadata_index(index) + + return True + + def update_metadata(self, model_id: str, metadata: Dict[str, Any]) -> bool: + """ + Update metadata for a registered model. + + Args: + model_id (str): Model ID to update + metadata (Dict[str, Any]): New metadata to add/update + + Returns: + bool: True if successful, False otherwise + """ + model_info = self.get_model(model_id) + if not model_info: + return False + + # Update metadata + model_info.update(metadata) + + # Save updated metadata + metadata_file = self.metadata_path / f"{model_id}.json" + with open(metadata_file, 'w') as f: + json.dump(model_info, f, indent=2) + + return True \ No newline at end of file diff --git a/igel/igel/model_versioning.py b/igel/igel/model_versioning.py new file mode 100644 index 0000000..343acfd --- /dev/null +++ b/igel/igel/model_versioning.py @@ -0,0 +1,674 @@ +""" +Enhanced Model Versioning System for Igel. + +This module provides advanced model versioning capabilities including: +- Model lineage tracking +- Metadata management +- Experiment integration +- Model comparison and visualization +- Deployment tracking +""" + +import os +import json +import datetime +import uuid +import hashlib +from pathlib import Path +from typing import Dict, List, Optional, Any, Union +import pandas as pd +import joblib +import logging +from dataclasses import dataclass, asdict +import sqlite3 +import shutil + +logger = logging.getLogger(__name__) + + +@dataclass +class ModelVersion: + """Model version information.""" + version_id: str + model_name: str + version: str + created_at: str + created_by: str + description: str + tags: Dict[str, str] + model_path: str + model_type: str + model_algorithm: str + parameters: Dict[str, Any] + metrics: Dict[str, float] + dataset_info: Dict[str, Any] + dependencies: Dict[str, str] + lineage: Dict[str, Any] + status: str # DRAFT, STAGING, PRODUCTION, ARCHIVED + + +@dataclass +class ModelDeployment: + """Model deployment information.""" + deployment_id: str + model_version_id: str + environment: str + deployed_at: str + deployed_by: str + status: str # ACTIVE, INACTIVE, FAILED + endpoint: Optional[str] + performance_metrics: Dict[str, float] + + +class ModelVersioning: + """ + Enhanced model versioning system with lineage tracking. + + Provides functionality for: + - Model versioning with semantic versioning + - Lineage tracking from experiments + - Metadata management + - Deployment tracking + - Model comparison and visualization + """ + + def __init__(self, versioning_path: str = "model_versions"): + """ + Initialize the model versioning system. + + Args: + versioning_path (str): Path to store versioned models + """ + self.versioning_path = Path(versioning_path) + self.versioning_path.mkdir(exist_ok=True) + + # Database for storing version metadata + self.db_path = self.versioning_path / "versions.db" + self._init_database() + + # Model storage + self.models_path = self.versioning_path / "models" + self.models_path.mkdir(exist_ok=True) + + # Deployments + self.deployments_path = self.versioning_path / "deployments" + self.deployments_path.mkdir(exist_ok=True) + + def _init_database(self): + """Initialize SQLite database for model versioning.""" + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + # Create model versions table + cursor.execute(''' + CREATE TABLE IF NOT EXISTS model_versions ( + version_id TEXT PRIMARY KEY, + model_name TEXT NOT NULL, + version TEXT NOT NULL, + created_at TEXT NOT NULL, + created_by TEXT NOT NULL, + description TEXT, + tags TEXT, + model_path TEXT NOT NULL, + model_type TEXT NOT NULL, + model_algorithm TEXT NOT NULL, + parameters TEXT, + metrics TEXT, + dataset_info TEXT, + dependencies TEXT, + lineage TEXT, + status TEXT NOT NULL, + UNIQUE(model_name, version) + ) + ''') + + # Create deployments table + cursor.execute(''' + CREATE TABLE IF NOT EXISTS deployments ( + deployment_id TEXT PRIMARY KEY, + model_version_id TEXT NOT NULL, + environment TEXT NOT NULL, + deployed_at TEXT NOT NULL, + deployed_by TEXT NOT NULL, + status TEXT NOT NULL, + endpoint TEXT, + performance_metrics TEXT, + FOREIGN KEY (model_version_id) REFERENCES model_versions (version_id) + ) + ''') + + # Create model lineage table + cursor.execute(''' + CREATE TABLE IF NOT EXISTS model_lineage ( + version_id TEXT NOT NULL, + parent_version_id TEXT, + experiment_id TEXT, + run_id TEXT, + lineage_type TEXT NOT NULL, + created_at TEXT NOT NULL, + FOREIGN KEY (version_id) REFERENCES model_versions (version_id), + FOREIGN KEY (parent_version_id) REFERENCES model_versions (version_id) + ) + ''') + + conn.commit() + conn.close() + + def create_version(self, + model_name: str, + model, + model_type: str, + model_algorithm: str, + parameters: Dict[str, Any], + metrics: Dict[str, float], + dataset_info: Dict[str, Any], + description: str = "", + tags: Optional[Dict[str, str]] = None, + dependencies: Optional[Dict[str, str]] = None, + parent_version_id: Optional[str] = None, + experiment_id: Optional[str] = None, + run_id: Optional[str] = None, + created_by: str = "user") -> str: + """ + Create a new model version. + + Args: + model_name (str): Name of the model + model: The model object to version + model_type (str): Type of model (regression, classification, etc.) + model_algorithm (str): Algorithm used + parameters (Dict[str, Any]): Model parameters + metrics (Dict[str, float]): Model performance metrics + dataset_info (Dict[str, Any]): Information about the training dataset + description (str): Version description + tags (Optional[Dict[str, str]]): Version tags + dependencies (Optional[Dict[str, str]]): Model dependencies + parent_version_id (Optional[str]): Parent version ID for lineage + experiment_id (Optional[str]): Associated experiment ID + run_id (Optional[str]): Associated run ID + created_by (str): User who created the version + + Returns: + str: Version ID + """ + # Generate version ID and determine version number + version_id = str(uuid.uuid4()) + version = self._get_next_version(model_name) + created_at = datetime.datetime.now().isoformat() + + # Create model directory + model_dir = self.models_path / version_id + model_dir.mkdir(exist_ok=True) + + # Save model + model_path = model_dir / "model.joblib" + joblib.dump(model, model_path) + + # Prepare lineage information + lineage = { + "parent_version_id": parent_version_id, + "experiment_id": experiment_id, + "run_id": run_id, + "lineage_type": "experiment" if experiment_id else "manual" + } + + # Create model version record + model_version = ModelVersion( + version_id=version_id, + model_name=model_name, + version=version, + created_at=created_at, + created_by=created_by, + description=description, + tags=tags or {}, + model_path=str(model_path), + model_type=model_type, + model_algorithm=model_algorithm, + parameters=parameters, + metrics=metrics, + dataset_info=dataset_info, + dependencies=dependencies or {}, + lineage=lineage, + status="DRAFT" + ) + + # Save to database + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + cursor.execute(''' + INSERT INTO model_versions + (version_id, model_name, version, created_at, created_by, description, + tags, model_path, model_type, model_algorithm, parameters, metrics, + dataset_info, dependencies, lineage, status) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + ''', ( + model_version.version_id, + model_version.model_name, + model_version.version, + model_version.created_at, + model_version.created_by, + model_version.description, + json.dumps(model_version.tags), + model_version.model_path, + model_version.model_type, + model_version.model_algorithm, + json.dumps(model_version.parameters), + json.dumps(model_version.metrics), + json.dumps(model_version.dataset_info), + json.dumps(model_version.dependencies), + json.dumps(model_version.lineage), + model_version.status + )) + + # Save lineage information + if parent_version_id or experiment_id or run_id: + cursor.execute(''' + INSERT INTO model_lineage + (version_id, parent_version_id, experiment_id, run_id, lineage_type, created_at) + VALUES (?, ?, ?, ?, ?, ?) + ''', ( + version_id, + parent_version_id, + experiment_id, + run_id, + lineage["lineage_type"], + created_at + )) + + conn.commit() + conn.close() + + logger.info(f"Created model version {version} for {model_name} with ID: {version_id}") + return version_id + + def _get_next_version(self, model_name: str) -> str: + """ + Get the next version number for a model. + + Args: + model_name (str): Model name + + Returns: + str: Next version number (e.g., "1.0.0", "1.1.0") + """ + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + SELECT version FROM model_versions + WHERE model_name = ? ORDER BY version DESC LIMIT 1 + ''', (model_name,)) + + row = cursor.fetchone() + conn.close() + + if row: + current_version = row[0] + # Simple version increment - in production, use semantic versioning + try: + major, minor, patch = map(int, current_version.split('.')) + return f"{major}.{minor}.{patch + 1}" + except: + return f"{current_version}.1" + else: + return "1.0.0" + + def get_version(self, version_id: str) -> Optional[ModelVersion]: + """ + Get model version by ID. + + Args: + version_id (str): Version ID + + Returns: + Optional[ModelVersion]: Model version if found, None otherwise + """ + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + SELECT version_id, model_name, version, created_at, created_by, description, + tags, model_path, model_type, model_algorithm, parameters, metrics, + dataset_info, dependencies, lineage, status + FROM model_versions WHERE version_id = ? + ''', (version_id,)) + + row = cursor.fetchone() + conn.close() + + if row: + return ModelVersion( + version_id=row[0], + model_name=row[1], + version=row[2], + created_at=row[3], + created_by=row[4], + description=row[5], + tags=json.loads(row[6]) if row[6] else {}, + model_path=row[7], + model_type=row[8], + model_algorithm=row[9], + parameters=json.loads(row[10]) if row[10] else {}, + metrics=json.loads(row[11]) if row[11] else {}, + dataset_info=json.loads(row[12]) if row[12] else {}, + dependencies=json.loads(row[13]) if row[13] else {}, + lineage=json.loads(row[14]) if row[14] else {}, + status=row[15] + ) + return None + + def list_versions(self, model_name: Optional[str] = None) -> List[ModelVersion]: + """ + List model versions, optionally filtered by model name. + + Args: + model_name (Optional[str]): Filter by model name + + Returns: + List[ModelVersion]: List of model versions + """ + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + if model_name: + cursor.execute(''' + SELECT version_id, model_name, version, created_at, created_by, description, + tags, model_path, model_type, model_algorithm, parameters, metrics, + dataset_info, dependencies, lineage, status + FROM model_versions WHERE model_name = ? ORDER BY created_at DESC + ''', (model_name,)) + else: + cursor.execute(''' + SELECT version_id, model_name, version, created_at, created_by, description, + tags, model_path, model_type, model_algorithm, parameters, metrics, + dataset_info, dependencies, lineage, status + FROM model_versions ORDER BY created_at DESC + ''') + + versions = [] + for row in cursor.fetchall(): + versions.append(ModelVersion( + version_id=row[0], + model_name=row[1], + version=row[2], + created_at=row[3], + created_by=row[4], + description=row[5], + tags=json.loads(row[6]) if row[6] else {}, + model_path=row[7], + model_type=row[8], + model_algorithm=row[9], + parameters=json.loads(row[10]) if row[10] else {}, + metrics=json.loads(row[11]) if row[11] else {}, + dataset_info=json.loads(row[12]) if row[12] else {}, + dependencies=json.loads(row[13]) if row[13] else {}, + lineage=json.loads(row[14]) if row[14] else {}, + status=row[15] + )) + + conn.close() + return versions + + def load_model(self, version_id: str): + """ + Load a model from version ID. + + Args: + version_id (str): Version ID + + Returns: + The loaded model + """ + version = self.get_version(version_id) + if not version: + raise ValueError(f"Model version {version_id} not found") + + if not Path(version.model_path).exists(): + raise FileNotFoundError(f"Model file not found: {version.model_path}") + + return joblib.load(version.model_path) + + def update_status(self, version_id: str, status: str): + """ + Update the status of a model version. + + Args: + version_id (str): Version ID + status (str): New status (DRAFT, STAGING, PRODUCTION, ARCHIVED) + """ + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + UPDATE model_versions SET status = ? WHERE version_id = ? + ''', (status, version_id)) + conn.commit() + conn.close() + + logger.info(f"Updated model version {version_id} status to {status}") + + def deploy_model(self, + version_id: str, + environment: str, + deployed_by: str = "user", + endpoint: Optional[str] = None) -> str: + """ + Deploy a model version to an environment. + + Args: + version_id (str): Version ID to deploy + environment (str): Target environment + deployed_by (str): User who deployed the model + endpoint (Optional[str]): Model endpoint URL + + Returns: + str: Deployment ID + """ + deployment_id = str(uuid.uuid4()) + deployed_at = datetime.datetime.now().isoformat() + + deployment = ModelDeployment( + deployment_id=deployment_id, + model_version_id=version_id, + environment=environment, + deployed_at=deployed_at, + deployed_by=deployed_by, + status="ACTIVE", + endpoint=endpoint, + performance_metrics={} + ) + + # Save to database + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + INSERT INTO deployments + (deployment_id, model_version_id, environment, deployed_at, deployed_by, status, endpoint, performance_metrics) + VALUES (?, ?, ?, ?, ?, ?, ?, ?) + ''', ( + deployment.deployment_id, + deployment.model_version_id, + deployment.environment, + deployment.deployed_at, + deployment.deployed_by, + deployment.status, + deployment.endpoint, + json.dumps(deployment.performance_metrics) + )) + conn.commit() + conn.close() + + # Update model status to PRODUCTION if deploying to production + if environment.lower() == "production": + self.update_status(version_id, "PRODUCTION") + + logger.info(f"Deployed model version {version_id} to {environment}") + return deployment_id + + def get_deployments(self, version_id: Optional[str] = None) -> List[ModelDeployment]: + """ + Get model deployments. + + Args: + version_id (Optional[str]): Filter by version ID + + Returns: + List[ModelDeployment]: List of deployments + """ + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + + if version_id: + cursor.execute(''' + SELECT deployment_id, model_version_id, environment, deployed_at, deployed_by, status, endpoint, performance_metrics + FROM deployments WHERE model_version_id = ? ORDER BY deployed_at DESC + ''', (version_id,)) + else: + cursor.execute(''' + SELECT deployment_id, model_version_id, environment, deployed_at, deployed_by, status, endpoint, performance_metrics + FROM deployments ORDER BY deployed_at DESC + ''') + + deployments = [] + for row in cursor.fetchall(): + deployments.append(ModelDeployment( + deployment_id=row[0], + model_version_id=row[1], + environment=row[2], + deployed_at=row[3], + deployed_by=row[4], + status=row[5], + endpoint=row[6], + performance_metrics=json.loads(row[7]) if row[7] else {} + )) + + conn.close() + return deployments + + def compare_versions(self, version_ids: List[str]) -> pd.DataFrame: + """ + Compare multiple model versions. + + Args: + version_ids (List[str]): List of version IDs to compare + + Returns: + pd.DataFrame: Comparison table + """ + versions = [] + for version_id in version_ids: + version = self.get_version(version_id) + if version: + versions.append(version) + + if not versions: + return pd.DataFrame() + + # Prepare comparison data + comparison_data = [] + for version in versions: + data = { + "version_id": version.version_id, + "model_name": version.model_name, + "version": version.version, + "created_at": version.created_at, + "created_by": version.created_by, + "status": version.status, + "model_type": version.model_type, + "model_algorithm": version.model_algorithm, + **version.metrics, + **{f"param_{k}": v for k, v in version.parameters.items()} + } + comparison_data.append(data) + + return pd.DataFrame(comparison_data) + + def get_lineage(self, version_id: str) -> Dict[str, Any]: + """ + Get the lineage of a model version. + + Args: + version_id (str): Version ID + + Returns: + Dict[str, Any]: Lineage information + """ + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute(''' + SELECT parent_version_id, experiment_id, run_id, lineage_type, created_at + FROM model_lineage WHERE version_id = ? + ''', (version_id,)) + + row = cursor.fetchone() + conn.close() + + if row: + return { + "parent_version_id": row[0], + "experiment_id": row[1], + "run_id": row[2], + "lineage_type": row[3], + "created_at": row[4] + } + return {} + + def export_version(self, version_id: str, output_path: str): + """ + Export a model version to a file. + + Args: + version_id (str): Version ID to export + output_path (str): Output file path + """ + version = self.get_version(version_id) + if not version: + raise ValueError(f"Model version {version_id} not found") + + # Copy model file + output_dir = Path(output_path).parent + output_dir.mkdir(exist_ok=True) + + model_file = output_dir / f"{version.model_name}_v{version.version}.joblib" + shutil.copy2(version.model_path, model_file) + + # Export metadata + metadata_file = output_dir / f"{version.model_name}_v{version.version}_metadata.json" + with open(metadata_file, 'w') as f: + json.dump(asdict(version), f, indent=2) + + logger.info(f"Exported model version {version_id} to {output_path}") + + def delete_version(self, version_id: str) -> bool: + """ + Delete a model version. + + Args: + version_id (str): Version ID to delete + + Returns: + bool: True if successful, False otherwise + """ + version = self.get_version(version_id) + if not version: + return False + + # Check if version is deployed + deployments = self.get_deployments(version_id) + if deployments: + logger.warning(f"Cannot delete version {version_id} - it has active deployments") + return False + + # Remove model files + model_dir = Path(version.model_path).parent + if model_dir.exists(): + shutil.rmtree(model_dir) + + # Remove from database + conn = sqlite3.connect(self.db_path) + cursor = conn.cursor() + cursor.execute('DELETE FROM model_versions WHERE version_id = ?', (version_id,)) + cursor.execute('DELETE FROM model_lineage WHERE version_id = ?', (version_id,)) + conn.commit() + conn.close() + + logger.info(f"Deleted model version {version_id}") + return True \ No newline at end of file diff --git a/igel/igel/model_watermark.py b/igel/igel/model_watermark.py new file mode 100644 index 0000000..23becdf --- /dev/null +++ b/igel/igel/model_watermark.py @@ -0,0 +1,19 @@ +""" +Model Watermarking Utilities + +- Digital watermarking for model ownership (placeholder) +- Watermark detection and verification (placeholder) +- Intellectual property protection (placeholder) +""" + +def add_watermark(model, watermark_info): + # Placeholder: Add real watermarking logic here + return model + +def detect_watermark(model): + # Placeholder: Add real detection logic here + return {"watermark_found": True, "info": "example"} + +def verify_watermark(model, expected_info): + # Placeholder: Add real verification logic here + return {"verified": True} diff --git a/igel/igel/model_watermarking.py b/igel/igel/model_watermarking.py new file mode 100644 index 0000000..23becdf --- /dev/null +++ b/igel/igel/model_watermarking.py @@ -0,0 +1,19 @@ +""" +Model Watermarking Utilities + +- Digital watermarking for model ownership (placeholder) +- Watermark detection and verification (placeholder) +- Intellectual property protection (placeholder) +""" + +def add_watermark(model, watermark_info): + # Placeholder: Add real watermarking logic here + return model + +def detect_watermark(model): + # Placeholder: Add real detection logic here + return {"watermark_found": True, "info": "example"} + +def verify_watermark(model, expected_info): + # Placeholder: Add real verification logic here + return {"verified": True} diff --git a/igel/igel/preprocessing.py b/igel/igel/preprocessing.py new file mode 100644 index 0000000..767699a --- /dev/null +++ b/igel/igel/preprocessing.py @@ -0,0 +1,142 @@ +import logging + +import numpy as np +import pandas as pd +from sklearn.impute import SimpleImputer +from sklearn.preprocessing import ( + LabelEncoder, + MinMaxScaler, + OneHotEncoder, + StandardScaler, +) +from tqdm import tqdm + +logging.basicConfig(format="%(levelname)s - %(message)s", level=logging.INFO) +logger = logging.getLogger(__name__) + + +def read_data_to_df(data_path: str, source_type: str = None, connection_string: str = None, sql_query: str = None, **read_data_options): + """ + Read data from various sources and convert it to a pandas dataframe. + Supports CSV, TXT, Excel, JSON, HTML, Parquet, and SQL. + """ + if not source_type: + file_ext = data_path.split(".")[-1] + if file_ext in ["csv", "txt"]: + source_type = "csv" + elif file_ext == "xlsx": + source_type = "excel" + elif file_ext == "json": + source_type = "json" + elif file_ext == "html": + source_type = "html" + elif file_ext == "parquet": + source_type = "parquet" + else: + raise ValueError(f"Unknown file extension: {file_ext}. Please specify source_type explicitly.") + + if source_type == "csv": + if "chunksize" in read_data_options: + return pd.concat([ + chunk for chunk in tqdm( + pd.read_csv(data_path, **read_data_options), + desc="Loading data" + ) + ]) + else: + return pd.read_csv(data_path, **read_data_options) + elif source_type == "excel": + return pd.read_excel(data_path, **read_data_options) + elif source_type == "json": + return pd.read_json(data_path, **read_data_options) + elif source_type == "html": + return pd.read_html(data_path, **read_data_options) + elif source_type == "parquet": + return pd.read_parquet(data_path, **read_data_options) + elif source_type == "sql": + if not connection_string or not sql_query: + raise ValueError("For SQL source, both connection_string and sql_query must be provided.") + import sqlalchemy + engine = sqlalchemy.create_engine(connection_string) + return pd.read_sql(sql_query, engine, **read_data_options) + else: + raise ValueError(f"Unsupported source_type: {source_type}") + + +def update_dataset_props(dataset_props: dict, default_dataset_props: dict): + for key1 in default_dataset_props.keys(): + if key1 in dataset_props.keys(): + for key2 in default_dataset_props[key1].keys(): + if key2 in dataset_props[key1].keys(): + default_dataset_props[key1][key2] = dataset_props[key1][ + key2 + ] + + return default_dataset_props + + +def handle_missing_values(df, fill_value=np.nan, strategy="mean"): + logger.info( + f"Check for missing values in the dataset ... \n" + f"{df.isna().sum()} \n " + f"{'-'*100}" + ) + + if strategy.lower() == "drop": + return df.dropna() + + cleaner = SimpleImputer(fill_value=fill_value, strategy=strategy) + cleaned = cleaner.fit_transform(df) + + return pd.DataFrame(cleaned, columns=df.columns) + + +def encode(df, encoding_type="onehotencoding", column=None): + if not encoding_type: + raise Exception( + f"encoding type should be -> oneHotEncoding or labelEncoding" + ) + + if encoding_type == "onehotencoding": + logger.info(f"performing a one hot encoding ...") + return pd.get_dummies(df), None + + elif encoding_type == "labelencoding": + if not column: + raise Exception( + "if you choose to label encode your data, " + "then you need to provide the column you want to encode from your dataset" + ) + logger.info(f"performing a label encoding ...") + encoder = LabelEncoder() + encoder.fit(df[column]) + classes_map = { + cls: int(lbl) + for (cls, lbl) in zip( + encoder.classes_, encoder.transform(encoder.classes_) + ) + } + logger.info(f"label encoding classes => {encoder.classes_}") + logger.info(f"classes map => {classes_map}") + df[column] = encoder.transform(df[column]) + return df, classes_map + + else: + raise Exception( + f"encoding type should be -> oneHotEncoding or labelEncoding" + ) + + +def normalize(x, y=None, method="standard"): + methods = ("minmax", "standard") + + if method not in methods: + raise Exception( + f"Please choose one of the available scaling methods => {methods}" + ) + logger.info(f"performing a {method} scaling ...") + scaler = MinMaxScaler() if method == "minmax" else StandardScaler() + if not y: + return scaler.fit_transform(X=x) + else: + return scaler.fit_transform(X=x, y=y) diff --git a/igel/igel/servers/__init__.py b/igel/igel/servers/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/igel/igel/servers/fastapi_server.py b/igel/igel/servers/fastapi_server.py new file mode 100644 index 0000000..ef86929 --- /dev/null +++ b/igel/igel/servers/fastapi_server.py @@ -0,0 +1,90 @@ +import logging +import os +from pathlib import Path + +import pandas as pd +import uvicorn +from fastapi import Body, FastAPI +from igel import Igel +from igel.configs import temp_post_req_data_path +from igel.constants import Constants + +try: + from .helper import remove_temp_data_file +except ImportError: + from igel.servers.helper import remove_temp_data_file + + +logger = logging.getLogger(__name__) + + +app = FastAPI() + + +@app.get("/") +async def just_for_testing(): + return {"success": True} + + +@app.post("/predict") +async def predict(data: dict = Body(...)): + """ + parse json data received from client, use pre-trained model to generate predictions and send them back to client + """ + try: + logger.info( + f"received request successfully, data will be parsed and used as inputs to generate predictions" + ) + + # convert values to list in order to convert it later to pandas dataframe + data = { + k: [v] if not isinstance(v, list) else v for k, v in data.items() + } + + # convert received data to dataframe + df = pd.DataFrame(data, index=None) + df.to_csv(temp_post_req_data_path, index=False) + + # use igel to generate predictions + model_resutls_path = os.environ.get(Constants.model_results_path) + logger.info(f"model_results path: {model_resutls_path}") + + if not model_resutls_path: + logger.warning( + f"Please provide path to the model_results directory generated by igel using the cli!" + ) + else: + model_path = Path(model_resutls_path) / Constants.model_file + description_file = ( + Path(model_resutls_path) / Constants.description_file + ) + prediction_file = ( + Path(model_resutls_path) / Constants.prediction_file + ) + + res = Igel( + cmd="predict", + data_path=str(temp_post_req_data_path), + model_path=model_path, + description_file=description_file, + prediction_file=prediction_file, + ) + + # remove temp file: + remove_temp_data_file(temp_post_req_data_path) + + logger.info("sending predictions back to client...") + return {"prediction": res.predictions.to_numpy().tolist()} + + except FileNotFoundError as ex: + remove_temp_data_file(temp_post_req_data_path) + logger.exception(ex) + + +def run(**kwargs): + + uvicorn.run(app, **kwargs) + + +if __name__ == "__main__": + run() diff --git a/igel/igel/servers/helper.py b/igel/igel/servers/helper.py new file mode 100644 index 0000000..9adee9a --- /dev/null +++ b/igel/igel/servers/helper.py @@ -0,0 +1,14 @@ +import logging +import os + +logger = logging.getLogger(__name__) + + +def remove_temp_data_file(f): + """ + remove temporary file, where request payload has been stored in order to be used by igel to generate predictions + """ + # remove temp file: + if os.path.exists(f): + logger.info(f"removing temporary file: {f}") + os.remove(f) diff --git a/igel/igel/smart_grid.py b/igel/igel/smart_grid.py new file mode 100644 index 0000000..0cab241 --- /dev/null +++ b/igel/igel/smart_grid.py @@ -0,0 +1,19 @@ +""" +Smart Grid Management Utilities + +- Load balancing optimization (placeholder) +- Energy demand prediction (placeholder) +- Grid stability analysis (placeholder) +""" + +def optimize_load_balancing(grid_data): + # Placeholder: Add real optimization logic here + return {"strategy": "even_distribution", "efficiency": 0.95} + +def predict_energy_demand(historical_data): + # Placeholder: Add real prediction logic here + return {"predicted_demand": 1000, "unit": "kWh"} + +def analyze_grid_stability(grid_metrics): + # Placeholder: Add real stability analysis here + return {"stability_score": 0.98, "status": "stable"} diff --git a/igel/igel/space_mission.py b/igel/igel/space_mission.py new file mode 100644 index 0000000..4613c20 --- /dev/null +++ b/igel/igel/space_mission.py @@ -0,0 +1,77 @@ +""" +Space Mission Planning Utilities + +- Trajectory optimization (placeholder) +- Resource allocation (placeholder) +- Mission simulation (placeholder) +""" + +import numpy as np + +def optimize_trajectory(start_point, end_point, constraints): + """ + Optimize spacecraft trajectory between two points. + + Args: + start_point: Starting coordinates [x, y, z] + end_point: Destination coordinates [x, y, z] + constraints: Dictionary of mission constraints + + Returns: + Dictionary with optimized path and efficiency metrics + """ + # Placeholder: Add real trajectory optimization logic here + # This could use orbital mechanics, Hohmann transfers, etc. + return { + "optimized_path": [start_point, end_point], + "fuel_efficiency": 0.85, + "travel_time": 365, # days + "delta_v": 5000 # m/s + } + +def allocate_resources(mission_goals, available_resources): + """ + Allocate resources for space mission objectives. + + Args: + mission_goals: List of mission objectives + available_resources: Dictionary of available resources + + Returns: + Dictionary with allocated resources and efficiency + """ + # Placeholder: Add real resource allocation logic here + # This could use optimization algorithms for resource distribution + return { + "allocated_resources": { + "fuel": 1000, # kg + "power": 500, # watts + "crew_time": 2000 # hours + }, + "efficiency": 0.92, + "mission_duration": 730 # days + } + +def simulate_mission(mission_plan, environment_params): + """ + Simulate mission execution with given parameters. + + Args: + mission_plan: Dictionary containing mission details + environment_params: Dictionary of environmental conditions + + Returns: + Dictionary with simulation results and success probability + """ + # Placeholder: Add real mission simulation logic here + # This could use Monte Carlo simulations, risk assessment, etc. + return { + "success_probability": 0.87, + "events": [ + {"time": 0, "event": "Mission start"}, + {"time": 30, "event": "Orbital insertion"}, + {"time": 365, "event": "Primary mission complete"} + ], + "risk_factors": ["radiation", "micro-meteoroids", "system_failures"], + "contingency_plans": ["backup_systems", "emergency_protocols"] + } \ No newline at end of file diff --git a/igel/igel/synthetic_data_generator.py b/igel/igel/synthetic_data_generator.py new file mode 100644 index 0000000..c0a5d06 --- /dev/null +++ b/igel/igel/synthetic_data_generator.py @@ -0,0 +1,48 @@ +""" +Synthetic Data Generator for Igel. + +This module provides functionality to generate synthetic datasets for testing and examples. +Addresses GitHub issue #285 - Add Support for Synthetic Data Generation. +""" + +import numpy as np +import pandas as pd +from sklearn.datasets import make_classification, make_regression +import logging + +logger = logging.getLogger(__name__) + + +class SyntheticDataGenerator: + """Generate synthetic datasets for testing and examples.""" + + def __init__(self, random_state: int = 42): + self.random_state = random_state + np.random.seed(random_state) + + def generate_classification_data(self, n_samples: int = 1000, n_features: int = 20) -> pd.DataFrame: + """Generate synthetic classification dataset.""" + X, y = make_classification(n_samples=n_samples, n_features=n_features, random_state=self.random_state) + feature_names = [f"feature_{i}" for i in range(n_features)] + df = pd.DataFrame(X, columns=feature_names) + df["target"] = y + return df + + def generate_regression_data(self, n_samples: int = 1000, n_features: int = 20) -> pd.DataFrame: + """Generate synthetic regression dataset.""" + X, y = make_regression(n_samples=n_samples, n_features=n_features, random_state=self.random_state) + feature_names = [f"feature_{i}" for i in range(n_features)] + df = pd.DataFrame(X, columns=feature_names) + df["target"] = y + return df + + +def generate_sample_dataset(dataset_type: str = "classification", **kwargs) -> pd.DataFrame: + """Quick function to generate sample datasets.""" + generator = SyntheticDataGenerator() + if dataset_type == "classification": + return generator.generate_classification_data(**kwargs) + elif dataset_type == "regression": + return generator.generate_regression_data(**kwargs) + else: + raise ValueError(f"Unknown dataset type: {dataset_type}") diff --git a/igel/igel/utils.py b/igel/igel/utils.py new file mode 100644 index 0000000..f44201d --- /dev/null +++ b/igel/igel/utils.py @@ -0,0 +1,227 @@ +import json +import logging + +import joblib +import pandas as pd +import yaml +from igel.configs import configs +from igel.data import metrics_dict, models_dict + +logger = logging.getLogger(__name__) + + +def create_yaml(data, f): + try: + with open(f, "w") as yf: + yaml.dump(data, yf, default_flow_style=False) + except yaml.YAMLError as exc: + logger.exception(exc) + return False + else: + return True + + +def read_yaml(f): + with open(f) as stream: + try: + res = yaml.safe_load(stream) + except yaml.YAMLError as exc: + logger.exception(exc) + else: + return res + + +def read_json(f): + try: + with open(f) as file: + data = json.load(file) + except Exception as e: + logger.exception(e.args) + else: + return data + + +def extract_params(config): + assert ( + "model" in config.keys() + ), "model parameters need to be provided in the yaml file" + assert ( + "target" in config.keys() + ), "target variable needs to be provided in the yaml file" + model_params = config.get("model") + model_type = model_params.get("type") + algorithm = model_params.get("algorithm") + target = config.get("target") + + if any(not item for item in [model_type, target, algorithm]): + raise Exception("parameters in the model yaml file cannot be None") + else: + return model_type, target, algorithm + + +def _reshape(arr): + if len(arr.shape) <= 1: + arr = arr.reshape(-1, 1) + return arr + + +def load_trained_model(f: str = ""): + """ + load a saved model from file + @param f: path to model + @return: loaded model + """ + try: + if not f: + logger.info(f"result path: {configs.get('results_path')} ") + logger.info( + f"loading model form {configs.get('default_model_path')} " + ) + with open(configs.get("default_model_path"), "rb") as _model: + model = joblib.load(_model) + else: + logger.info(f"loading from {f}") + with open(f, "rb") as _model: + model = joblib.load(_model) + return model + except FileNotFoundError: + logger.error(f"File not found in {configs.get('default_model_path')}") + + +def load_train_configs(f=""): + """ + load train configurations from model_results/descriptions.json + """ + try: + if not f: + logger.info( + f"loading descriptions.json form {configs.get('description_file')} " + ) + with open(configs.get("description_file"), "rb") as desc_file: + training_config = json.load(desc_file) + else: + with open(f, "rb") as desc_file: + training_config = json.load(desc_file) + return training_config + + except FileNotFoundError as e: + logger.error(f"File not found: {e}") + except Exception as e: + logger.error(e) + + +def get_expected_scaling_method(training_config): + """ + get expected scaling method from the parsed training configuration (description.json) + """ + dataset_props = training_config.get("dataset_props") + if not dataset_props: + return + preprocess_options = dataset_props.get("preprocess") + if not preprocess_options: + return + scaling_options = preprocess_options.get("scale") + if not scaling_options: + return + return scaling_options.get("method") + + +def show_model_info(model_name: str, model_type: str): + if not model_name: + print(f"Please enter a supported model") + print_models_overview() + else: + if not model_type: + print( + f"Please enter a type argument to get help on the chosen model\n" + f"type can be whether regression, classification or clustering \n" + ) + print_models_overview() + return + if model_type not in ("regression", "classification", "clustering"): + raise Exception( + f"{model_type} is not supported! \n" + f"model_type need to be regression, classification or clustering" + ) + + models = models_dict.get(model_type) + model_data = models.get(model_name) + model, link, *cv_class = model_data.values() + print( + f"model type: {model_type} \n" + f"model name: {model_name} \n" + f"sklearn model class: {model.__name__} \n" + f"{'-' * 60}\n" + f"You can click the link below to know more about the optional arguments\n" + f"that you can use with your chosen model ({model_name}).\n" + f"You can provide these optional arguments in the yaml file if you want to use them.\n" + f"link:\n{link} \n" + ) + + +def tableize(df): + """ + pretty-print a dataframe as table + """ + if not isinstance(df, pd.DataFrame): + return + df_columns = df.columns.tolist() + max_len_in_lst = lambda lst: len(sorted(lst, reverse=True, key=len)[0]) + align_center = ( + lambda st, sz: "{0}{1}{0}".format(" " * (1 + (sz - len(st)) // 2), st)[ + :sz + ] + if len(st) < sz + else st + ) + align_right = ( + lambda st, sz: "{}{} ".format(" " * (sz - len(st) - 1), st) + if len(st) < sz + else st + ) + max_col_len = max_len_in_lst(df_columns) + max_val_len_for_col = { + col: max_len_in_lst(df.iloc[:, idx].astype("str")) + for idx, col in enumerate(df_columns) + } + col_sizes = { + col: 2 + max(max_val_len_for_col.get(col, 0), max_col_len) + for col in df_columns + } + build_hline = lambda row: "+".join( + ["-" * col_sizes[col] for col in row] + ).join(["+", "+"]) + build_data = lambda row, align: "|".join( + [ + align(str(val), col_sizes[df_columns[idx]]) + for idx, val in enumerate(row) + ] + ).join(["|", "|"]) + hline = build_hline(df_columns) + out = [hline, build_data(df_columns, align_center), hline] + for _, row in df.iterrows(): + out.append(build_data(row.tolist(), align_right)) + out.append(hline) + return "\n".join(out) + + +def print_models_overview(): + print(f"\nIgel's supported models overview: \n") + reg_algs = list(models_dict.get("regression").keys()) + clf_algs = list(models_dict.get("classification").keys()) + cluster_algs = list(models_dict.get("clustering").keys()) + df_algs = ( + pd.DataFrame.from_dict( + { + "regression": reg_algs, + "classification": clf_algs, + "clustering": cluster_algs, + }, + orient="index", + ) + .transpose() + .fillna("----") + ) + + df = tableize(df_algs) + print(df) diff --git a/igel/notebooks/igel_quickstart.ipynb b/igel/notebooks/igel_quickstart.ipynb new file mode 100644 index 0000000..c14d386 --- /dev/null +++ b/igel/notebooks/igel_quickstart.ipynb @@ -0,0 +1,104 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# igel Quickstart Tutorial\n", + "\n", + "Welcome! This notebook will walk you through a basic igel workflow: loading data, configuring a model, training, and evaluating." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "vscode": { + "languageId": "plaintext" + } + }, + "outputs": [], + "source": [ + "!pip install igel" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "vscode": { + "languageId": "plaintext" + } + }, + "outputs": [], + "source": [ + "import pandas as pd\n", + "\n", + "# Create a simple dataset\n", + "df = pd.DataFrame({\n", + " \"feature1\": [1, 2, 3, 4, 5],\n", + " \"feature2\": [5, 4, 3, 2, 1],\n", + " \"label\": [0, 1, 0, 1, 0]\n", + "})\n", + "df.to_csv(\"train.csv\", index=False)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "vscode": { + "languageId": "plaintext" + } + }, + "outputs": [], + "source": [ + "config = \"\"\"\n", + "input: train.csv\n", + "target: label\n", + "model:\n", + " type: classification\n", + " algorithm: RandomForestClassifier\n", + " params:\n", + " n_estimators: 10\n", + " max_depth: 3\n", + "\"\"\"\n", + "with open(\"config.yaml\", \"w\") as f:\n", + " f.write(config)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "vscode": { + "languageId": "plaintext" + } + }, + "outputs": [], + "source": [ + "!igel fit --data_path=train.csv --yaml_path=config.yaml" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "vscode": { + "languageId": "plaintext" + } + }, + "outputs": [], + "source": [ + "# You can add more cells to show evaluation or prediction steps as needed." + ] + } + ], + "metadata": { + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/igel/poetry.lock b/igel/poetry.lock new file mode 100644 index 0000000..5e74b53 --- /dev/null +++ b/igel/poetry.lock @@ -0,0 +1,2962 @@ +[[package]] +name = "absl-py" +version = "1.0.0" +description = "Abseil Python Common Libraries, see https://github.com/abseil/abseil-py." +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +six = "*" + +[[package]] +name = "alabaster" +version = "0.7.12" +description = "A configurable sidebar-enabled Sphinx theme" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "appnope" +version = "0.1.2" +description = "Disable App Nap on macOS >= 10.9" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "asgiref" +version = "3.4.1" +description = "ASGI specs, helper code, and adapters" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.extras] +tests = ["pytest", "pytest-asyncio", "mypy (>=0.800)"] + +[[package]] +name = "astroid" +version = "2.9.2" +description = "An abstract syntax tree for Python with inference support." +category = "dev" +optional = false +python-versions = ">=3.6.2" + +[package.dependencies] +lazy-object-proxy = ">=1.4.0" +typing-extensions = {version = ">=3.10", markers = "python_version < \"3.10\""} +wrapt = ">=1.11,<1.14" + +[[package]] +name = "astunparse" +version = "1.6.3" +description = "An AST unparser for Python" +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +six = ">=1.6.1,<2.0" + +[[package]] +name = "atomicwrites" +version = "1.4.0" +description = "Atomic file writes." +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[[package]] +name = "attrs" +version = "21.4.0" +description = "Classes Without Boilerplate" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" + +[package.extras] +dev = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "zope.interface", "furo", "sphinx", "sphinx-notfound-page", "pre-commit", "cloudpickle"] +docs = ["furo", "sphinx", "zope.interface", "sphinx-notfound-page"] +tests = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "zope.interface", "cloudpickle"] +tests_no_zope = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "cloudpickle"] + +[[package]] +name = "autokeras" +version = "1.0.16.post1" +description = "AutoML for deep learning" +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +keras-tuner = ">=1.0.2,<1.1" +packaging = "*" +pandas = "*" +scikit-learn = "*" +tensorflow = ">=2.3.0,<2.6" + +[package.extras] +tests = ["pytest (>=4.4.0)", "flake8", "black", "isort", "pytest-xdist", "pytest-cov", "coverage", "typeguard (>=2,<2.11.0)", "typedapi (>=0.2,<0.3)"] + +[[package]] +name = "babel" +version = "2.9.1" +description = "Internationalization utilities" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[package.dependencies] +pytz = ">=2015.7" + +[[package]] +name = "backcall" +version = "0.2.0" +description = "Specifications for callback functions passed in to an API" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "bandit" +version = "1.7.1" +description = "Security oriented static analyser for python code." +category = "dev" +optional = false +python-versions = ">=3.5" + +[package.dependencies] +colorama = {version = ">=0.3.9", markers = "platform_system == \"Windows\""} +GitPython = ">=1.0.1" +PyYAML = ">=5.3.1" +stevedore = ">=1.20.0" + +[[package]] +name = "black" +version = "21.12b0" +description = "The uncompromising code formatter." +category = "dev" +optional = false +python-versions = ">=3.6.2" + +[package.dependencies] +click = ">=7.1.2" +mypy-extensions = ">=0.4.3" +pathspec = ">=0.9.0,<1" +platformdirs = ">=2" +tomli = ">=0.2.6,<2.0.0" +typing-extensions = [ + {version = ">=3.10.0.0", markers = "python_version < \"3.10\""}, + {version = "!=3.10.0.1", markers = "python_version >= \"3.10\""}, +] + +[package.extras] +colorama = ["colorama (>=0.4.3)"] +d = ["aiohttp (>=3.7.4)"] +jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"] +python2 = ["typed-ast (>=1.4.3)"] +uvloop = ["uvloop (>=0.15.2)"] + +[[package]] +name = "bleach" +version = "4.1.0" +description = "An easy safelist-based HTML-sanitizing tool." +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +packaging = "*" +six = ">=1.9.0" +webencodings = "*" + +[[package]] +name = "bump2version" +version = "1.0.0" +description = "Version-bump your software with a single command!" +category = "dev" +optional = false +python-versions = ">=3.5" + +[[package]] +name = "cachetools" +version = "4.2.4" +description = "Extensible memoizing collections and decorators" +category = "main" +optional = false +python-versions = "~=3.5" + +[[package]] +name = "certifi" +version = "2021.10.8" +description = "Python package for providing Mozilla's CA Bundle." +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "cffi" +version = "1.15.0" +description = "Foreign Function Interface for Python calling C code." +category = "dev" +optional = false +python-versions = "*" + +[package.dependencies] +pycparser = "*" + +[[package]] +name = "cfgv" +version = "3.3.1" +description = "Validate configuration and produce human readable error messages." +category = "dev" +optional = false +python-versions = ">=3.6.1" + +[[package]] +name = "charset-normalizer" +version = "2.0.9" +description = "The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet." +category = "main" +optional = false +python-versions = ">=3.5.0" + +[package.extras] +unicode_backport = ["unicodedata2"] + +[[package]] +name = "click" +version = "8.0.3" +description = "Composable command line interface toolkit" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +colorama = {version = "*", markers = "platform_system == \"Windows\""} + +[[package]] +name = "colorama" +version = "0.4.4" +description = "Cross-platform colored terminal text." +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" + +[[package]] +name = "coverage" +version = "5.2.1" +description = "Code coverage measurement for Python" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <4" + +[package.extras] +toml = ["toml"] + +[[package]] +name = "cryptography" +version = "36.0.1" +description = "cryptography is a package which provides cryptographic recipes and primitives to Python developers." +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +cffi = ">=1.12" + +[package.extras] +docs = ["sphinx (>=1.6.5,!=1.8.0,!=3.1.0,!=3.1.1)", "sphinx-rtd-theme"] +docstest = ["pyenchant (>=1.6.11)", "twine (>=1.12.0)", "sphinxcontrib-spelling (>=4.0.1)"] +pep8test = ["black", "flake8", "flake8-import-order", "pep8-naming"] +sdist = ["setuptools_rust (>=0.11.4)"] +ssh = ["bcrypt (>=3.1.5)"] +test = ["pytest (>=6.2.0)", "pytest-cov", "pytest-subtests", "pytest-xdist", "pretend", "iso8601", "pytz", "hypothesis (>=1.11.4,!=3.79.2)"] + +[[package]] +name = "darglint" +version = "1.8.1" +description = "A utility for ensuring Google-style docstrings stay up to date with the source code." +category = "dev" +optional = false +python-versions = ">=3.6,<4.0" + +[[package]] +name = "decorator" +version = "5.1.0" +description = "Decorators for Humans" +category = "main" +optional = false +python-versions = ">=3.5" + +[[package]] +name = "distlib" +version = "0.3.4" +description = "Distribution utilities" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "docutils" +version = "0.18.1" +description = "Docutils -- Python Documentation Utilities" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" + +[[package]] +name = "dparse" +version = "0.5.1" +description = "A parser for Python dependency files" +category = "dev" +optional = false +python-versions = ">=3.5" + +[package.dependencies] +packaging = "*" +pyyaml = "*" +toml = "*" + +[package.extras] +pipenv = ["pipenv"] + +[[package]] +name = "fastapi" +version = "0.65.3" +description = "FastAPI framework, high performance, easy to learn, fast to code, ready for production" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +pydantic = ">=1.6.2,<1.7 || >1.7,<1.7.1 || >1.7.1,<1.7.2 || >1.7.2,<1.7.3 || >1.7.3,<1.8 || >1.8,<1.8.1 || >1.8.1,<2.0.0" +starlette = "0.14.2" + +[package.extras] +all = ["requests (>=2.24.0,<3.0.0)", "aiofiles (>=0.5.0,<0.6.0)", "jinja2 (>=2.11.2,<3.0.0)", "python-multipart (>=0.0.5,<0.0.6)", "itsdangerous (>=1.1.0,<2.0.0)", "pyyaml (>=5.3.1,<6.0.0)", "graphene (>=2.1.8,<3.0.0)", "ujson (>=4.0.1,<5.0.0)", "orjson (>=3.2.1,<4.0.0)", "email_validator (>=1.1.1,<2.0.0)", "uvicorn[standard] (>=0.12.0,<0.14.0)", "async_exit_stack (>=1.0.1,<2.0.0)", "async_generator (>=1.10,<2.0.0)"] +dev = ["python-jose[cryptography] (>=3.1.0,<4.0.0)", "passlib[bcrypt] (>=1.7.2,<2.0.0)", "autoflake (>=1.3.1,<2.0.0)", "flake8 (>=3.8.3,<4.0.0)", "uvicorn[standard] (>=0.12.0,<0.14.0)", "graphene (>=2.1.8,<3.0.0)"] +doc = ["mkdocs (>=1.1.2,<2.0.0)", "mkdocs-material (>=7.1.9,<8.0.0)", "markdown-include (>=0.6.0,<0.7.0)", "mkdocs-markdownextradata-plugin (>=0.1.7,<0.2.0)", "typer-cli (>=0.0.12,<0.0.13)", "pyyaml (>=5.3.1,<6.0.0)"] +test = ["pytest (==5.4.3)", "pytest-cov (==2.10.0)", "pytest-asyncio (>=0.14.0,<0.15.0)", "mypy (==0.812)", "flake8 (>=3.8.3,<4.0.0)", "black (==20.8b1)", "isort (>=5.0.6,<6.0.0)", "requests (>=2.24.0,<3.0.0)", "httpx (>=0.14.0,<0.15.0)", "email_validator (>=1.1.1,<2.0.0)", "sqlalchemy (>=1.3.18,<1.4.0)", "peewee (>=3.13.3,<4.0.0)", "databases[sqlite] (>=0.3.2,<0.4.0)", "orjson (>=3.2.1,<4.0.0)", "ujson (>=4.0.1,<5.0.0)", "async_exit_stack (>=1.0.1,<2.0.0)", "async_generator (>=1.10,<2.0.0)", "python-multipart (>=0.0.5,<0.0.6)", "aiofiles (>=0.5.0,<0.6.0)", "flask (>=1.1.2,<2.0.0)"] + +[[package]] +name = "filelock" +version = "3.4.2" +description = "A platform independent file lock." +category = "dev" +optional = false +python-versions = ">=3.7" + +[package.extras] +docs = ["furo (>=2021.8.17b43)", "sphinx (>=4.1)", "sphinx-autodoc-typehints (>=1.12)"] +testing = ["covdefaults (>=1.2.0)", "coverage (>=4)", "pytest (>=4)", "pytest-cov", "pytest-timeout (>=1.4.2)"] + +[[package]] +name = "flake8" +version = "3.8.3" +description = "the modular source code checker: pep8 pyflakes and co" +category = "dev" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7" + +[package.dependencies] +mccabe = ">=0.6.0,<0.7.0" +pycodestyle = ">=2.6.0a1,<2.7.0" +pyflakes = ">=2.2.0,<2.3.0" + +[[package]] +name = "gast" +version = "0.3.3" +description = "Python AST that abstracts the underlying Python version" +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[[package]] +name = "gitdb" +version = "4.0.9" +description = "Git Object Database" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +smmap = ">=3.0.1,<6" + +[[package]] +name = "gitpython" +version = "3.1.24" +description = "GitPython is a python library used to interact with Git repositories" +category = "dev" +optional = false +python-versions = ">=3.7" + +[package.dependencies] +gitdb = ">=4.0.1,<5" +typing-extensions = {version = ">=3.7.4.3", markers = "python_version < \"3.10\""} + +[[package]] +name = "google-auth" +version = "2.3.3" +description = "Google Authentication Library" +category = "main" +optional = false +python-versions = ">=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*" + +[package.dependencies] +cachetools = ">=2.0.0,<5.0" +pyasn1-modules = ">=0.2.1" +rsa = {version = ">=3.1.4,<5", markers = "python_version >= \"3.6\""} +six = ">=1.9.0" + +[package.extras] +aiohttp = ["requests (>=2.20.0,<3.0.0dev)", "aiohttp (>=3.6.2,<4.0.0dev)"] +pyopenssl = ["pyopenssl (>=20.0.0)"] +reauth = ["pyu2f (>=0.1.5)"] + +[[package]] +name = "google-auth-oauthlib" +version = "0.4.6" +description = "Google Authentication Library" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +google-auth = ">=1.0.0" +requests-oauthlib = ">=0.7.0" + +[package.extras] +tool = ["click (>=6.0.0)"] + +[[package]] +name = "google-pasta" +version = "0.2.0" +description = "pasta is an AST-based Python refactoring library" +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +six = "*" + +[[package]] +name = "grpcio" +version = "1.43.0" +description = "HTTP/2-based RPC framework" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +six = ">=1.5.2" + +[package.extras] +protobuf = ["grpcio-tools (>=1.43.0)"] + +[[package]] +name = "h11" +version = "0.12.0" +description = "A pure-Python, bring-your-own-I/O implementation of HTTP/1.1" +category = "main" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "h5py" +version = "2.10.0" +description = "Read and write HDF5 files from Python" +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +numpy = ">=1.7" +six = "*" + +[[package]] +name = "identify" +version = "2.4.1" +description = "File identification library for Python" +category = "dev" +optional = false +python-versions = ">=3.6.1" + +[package.extras] +license = ["ukkonen"] + +[[package]] +name = "idna" +version = "3.3" +description = "Internationalized Domain Names in Applications (IDNA)" +category = "main" +optional = false +python-versions = ">=3.5" + +[[package]] +name = "imagesize" +version = "1.3.0" +description = "Getting image size from png/jpeg/jpeg2000/gif file" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[[package]] +name = "importlib-metadata" +version = "1.7.0" +description = "Read metadata from Python packages" +category = "main" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7" + +[package.dependencies] +zipp = ">=0.5" + +[package.extras] +docs = ["sphinx", "rst.linker"] +testing = ["packaging", "pep517", "importlib-resources (>=1.3)"] + +[[package]] +name = "iniconfig" +version = "1.1.1" +description = "iniconfig: brain-dead simple config-ini parsing" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "ipython" +version = "7.30.1" +description = "IPython: Productive Interactive Computing" +category = "main" +optional = false +python-versions = ">=3.7" + +[package.dependencies] +appnope = {version = "*", markers = "sys_platform == \"darwin\""} +backcall = "*" +colorama = {version = "*", markers = "sys_platform == \"win32\""} +decorator = "*" +jedi = ">=0.16" +matplotlib-inline = "*" +pexpect = {version = ">4.3", markers = "sys_platform != \"win32\""} +pickleshare = "*" +prompt-toolkit = ">=2.0.0,<3.0.0 || >3.0.0,<3.0.1 || >3.0.1,<3.1.0" +pygments = "*" +traitlets = ">=4.2" + +[package.extras] +all = ["Sphinx (>=1.3)", "ipykernel", "ipyparallel", "ipywidgets", "nbconvert", "nbformat", "nose (>=0.10.1)", "notebook", "numpy (>=1.17)", "pygments", "qtconsole", "requests", "testpath"] +doc = ["Sphinx (>=1.3)"] +kernel = ["ipykernel"] +nbconvert = ["nbconvert"] +nbformat = ["nbformat"] +notebook = ["notebook", "ipywidgets"] +parallel = ["ipyparallel"] +qtconsole = ["qtconsole"] +test = ["nose (>=0.10.1)", "requests", "testpath", "pygments", "nbformat", "ipykernel", "numpy (>=1.17)"] + +[[package]] +name = "isort" +version = "5.10.1" +description = "A Python utility / library to sort Python imports." +category = "dev" +optional = false +python-versions = ">=3.6.1,<4.0" + +[package.extras] +pipfile_deprecated_finder = ["pipreqs", "requirementslib"] +requirements_deprecated_finder = ["pipreqs", "pip-api"] +colors = ["colorama (>=0.4.3,<0.5.0)"] +plugins = ["setuptools"] + +[[package]] +name = "jedi" +version = "0.18.1" +description = "An autocompletion tool for Python that can be used for text editors." +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +parso = ">=0.8.0,<0.9.0" + +[package.extras] +qa = ["flake8 (==3.8.3)", "mypy (==0.782)"] +testing = ["Django (<3.1)", "colorama", "docopt", "pytest (<7.0.0)"] + +[[package]] +name = "jeepney" +version = "0.7.1" +description = "Low-level, pure Python DBus protocol wrapper." +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.extras] +test = ["pytest", "pytest-trio", "pytest-asyncio", "testpath", "trio", "async-timeout"] +trio = ["trio", "async-generator"] + +[[package]] +name = "jinja2" +version = "3.0.3" +description = "A very fast and expressive template engine." +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +MarkupSafe = ">=2.0" + +[package.extras] +i18n = ["Babel (>=2.7)"] + +[[package]] +name = "joblib" +version = "0.16.0" +description = "Lightweight pipelining: using Python functions as pipeline jobs." +category = "main" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "keras-preprocessing" +version = "1.1.2" +description = "Easy data preprocessing and data augmentation for deep learning models" +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +numpy = ">=1.9.1" +six = ">=1.9.0" + +[package.extras] +image = ["scipy (>=0.14)", "Pillow (>=5.2.0)"] +pep8 = ["flake8"] +tests = ["pandas", "pillow", "tensorflow", "keras", "pytest", "pytest-xdist", "pytest-cov"] + +[[package]] +name = "keras-tuner" +version = "1.0.4" +description = "Hypertuner for Keras" +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +ipython = "*" +kt-legacy = "*" +numpy = "*" +packaging = "*" +requests = "*" +scipy = "*" +tensorboard = "*" + +[package.extras] +tests = ["pytest", "flake8", "isort", "black", "pandas", "portpicker", "pytest-xdist", "pytest-cov", "scikit-learn"] + +[[package]] +name = "keyring" +version = "22.3.0" +description = "Store and access your passwords safely." +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +jeepney = {version = ">=0.4.2", markers = "sys_platform == \"linux\""} +pywin32-ctypes = {version = "<0.1.0 || >0.1.0,<0.1.1 || >0.1.1", markers = "sys_platform == \"win32\""} +SecretStorage = {version = ">=3.2", markers = "sys_platform == \"linux\""} + +[package.extras] +docs = ["sphinx", "jaraco.packaging (>=8.2)", "rst.linker (>=1.9)"] +testing = ["pytest (>=4.6)", "pytest-checkdocs (>=1.2.3)", "pytest-flake8", "pytest-cov", "pytest-enabler", "pytest-black (>=0.3.7)", "pytest-mypy"] + +[[package]] +name = "kt-legacy" +version = "1.0.4" +description = "Legacy import names for Keras Tuner" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "lazy-object-proxy" +version = "1.7.1" +description = "A fast and thorough lazy object proxy." +category = "dev" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "markdown" +version = "3.3.5" +description = "Python implementation of Markdown." +category = "main" +optional = false +python-versions = ">=3.6" + +[package.extras] +testing = ["coverage", "pyyaml"] + +[[package]] +name = "markupsafe" +version = "2.0.1" +description = "Safely add untrusted strings to HTML/XML markup." +category = "dev" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "matplotlib-inline" +version = "0.1.3" +description = "Inline Matplotlib backend for Jupyter" +category = "main" +optional = false +python-versions = ">=3.5" + +[package.dependencies] +traitlets = "*" + +[[package]] +name = "mccabe" +version = "0.6.1" +description = "McCabe checker, plugin for flake8" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "more-itertools" +version = "8.12.0" +description = "More routines for operating on iterables, beyond itertools" +category = "dev" +optional = false +python-versions = ">=3.5" + +[[package]] +name = "mypy" +version = "0.902" +description = "Optional static typing for Python" +category = "dev" +optional = false +python-versions = ">=3.5" + +[package.dependencies] +mypy-extensions = ">=0.4.3,<0.5.0" +toml = "*" +typing-extensions = ">=3.7.4" + +[package.extras] +dmypy = ["psutil (>=4.0)"] +python2 = ["typed-ast (>=1.4.0,<1.5.0)"] + +[[package]] +name = "mypy-extensions" +version = "0.4.3" +description = "Experimental type system extensions for programs checked with the mypy typechecker." +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "nodeenv" +version = "1.6.0" +description = "Node.js virtual environment builder" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "numpy" +version = "1.18.5" +description = "NumPy is the fundamental package for array computing with Python." +category = "main" +optional = false +python-versions = ">=3.5" + +[[package]] +name = "oauthlib" +version = "3.1.1" +description = "A generic, spec-compliant, thorough implementation of the OAuth request-signing logic" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.extras] +rsa = ["cryptography (>=3.0.0,<4)"] +signals = ["blinker (>=1.4.0)"] +signedtoken = ["cryptography (>=3.0.0,<4)", "pyjwt (>=2.0.0,<3)"] + +[[package]] +name = "onnx" +version = "1.10.2" +description = "Open Neural Network Exchange" +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +numpy = ">=1.16.6" +protobuf = "*" +six = "*" +typing-extensions = ">=3.6.2.1" + +[package.extras] +mypy = ["mypy (==0.600)"] + +[[package]] +name = "onnxconverter-common" +version = "1.9.0" +description = "ONNX Converter and Optimization Tools" +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +numpy = "*" +onnx = "*" +protobuf = "*" + +[[package]] +name = "opt-einsum" +version = "3.3.0" +description = "Optimizing numpys einsum function" +category = "main" +optional = false +python-versions = ">=3.5" + +[package.dependencies] +numpy = ">=1.7" + +[package.extras] +docs = ["sphinx (==1.2.3)", "sphinxcontrib-napoleon", "sphinx-rtd-theme", "numpydoc"] +tests = ["pytest", "pytest-cov", "pytest-pep8"] + +[[package]] +name = "packaging" +version = "21.3" +description = "Core utilities for Python packages" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +pyparsing = ">=2.0.2,<3.0.5 || >3.0.5" + +[[package]] +name = "pandas" +version = "1.1.1" +description = "Powerful data structures for data analysis, time series, and statistics" +category = "main" +optional = false +python-versions = ">=3.6.1" + +[package.dependencies] +numpy = ">=1.15.4" +python-dateutil = ">=2.7.3" +pytz = ">=2017.2" + +[package.extras] +test = ["pytest (>=4.0.2)", "pytest-xdist", "hypothesis (>=3.58)"] + +[[package]] +name = "parso" +version = "0.8.3" +description = "A Python Parser" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.extras] +qa = ["flake8 (==3.8.3)", "mypy (==0.782)"] +testing = ["docopt", "pytest (<6.0.0)"] + +[[package]] +name = "pathspec" +version = "0.9.0" +description = "Utility library for gitignore style pattern matching of file paths." +category = "dev" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7" + +[[package]] +name = "pathtools" +version = "0.1.2" +description = "File system general utilities" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "pbr" +version = "5.8.0" +description = "Python Build Reasonableness" +category = "dev" +optional = false +python-versions = ">=2.6" + +[[package]] +name = "pexpect" +version = "4.8.0" +description = "Pexpect allows easy control of interactive console applications." +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +ptyprocess = ">=0.5" + +[[package]] +name = "pickleshare" +version = "0.7.5" +description = "Tiny 'shelve'-like database with concurrency support" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "pillow" +version = "8.4.0" +description = "Python Imaging Library (Fork)" +category = "main" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "pkginfo" +version = "1.8.2" +description = "Query metadatdata from sdists / bdists / installed packages." +category = "dev" +optional = false +python-versions = "*" + +[package.extras] +testing = ["coverage", "nose"] + +[[package]] +name = "platformdirs" +version = "2.4.1" +description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"." +category = "dev" +optional = false +python-versions = ">=3.7" + +[package.extras] +docs = ["Sphinx (>=4)", "furo (>=2021.7.5b38)", "proselint (>=0.10.2)", "sphinx-autodoc-typehints (>=1.12)"] +test = ["appdirs (==1.4.4)", "pytest (>=6)", "pytest-cov (>=2.7)", "pytest-mock (>=3.6)"] + +[[package]] +name = "pluggy" +version = "0.13.1" +description = "plugin and hook calling mechanisms for python" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[package.extras] +dev = ["pre-commit", "tox"] + +[[package]] +name = "pre-commit" +version = "2.16.0" +description = "A framework for managing and maintaining multi-language pre-commit hooks." +category = "dev" +optional = false +python-versions = ">=3.6.1" + +[package.dependencies] +cfgv = ">=2.0.0" +identify = ">=1.0.0" +nodeenv = ">=0.11.1" +pyyaml = ">=5.1" +toml = "*" +virtualenv = ">=20.0.8" + +[[package]] +name = "prompt-toolkit" +version = "3.0.24" +description = "Library for building powerful interactive command lines in Python" +category = "main" +optional = false +python-versions = ">=3.6.2" + +[package.dependencies] +wcwidth = "*" + +[[package]] +name = "protobuf" +version = "3.19.1" +description = "Protocol Buffers" +category = "main" +optional = false +python-versions = ">=3.5" + +[[package]] +name = "ptyprocess" +version = "0.7.0" +description = "Run a subprocess in a pseudo terminal" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "py" +version = "1.11.0" +description = "library with cross-python path, ini-parsing, io, code, log facilities" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" + +[[package]] +name = "pyasn1" +version = "0.4.8" +description = "ASN.1 types and codecs" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "pyasn1-modules" +version = "0.2.8" +description = "A collection of ASN.1-based protocols modules." +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +pyasn1 = ">=0.4.6,<0.5.0" + +[[package]] +name = "pycodestyle" +version = "2.6.0" +description = "Python style guide checker" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[[package]] +name = "pycparser" +version = "2.21" +description = "C parser in Python" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[[package]] +name = "pydantic" +version = "1.9.0" +description = "Data validation and settings management using python 3.6 type hinting" +category = "main" +optional = false +python-versions = ">=3.6.1" + +[package.dependencies] +typing-extensions = ">=3.7.4.3" + +[package.extras] +dotenv = ["python-dotenv (>=0.10.4)"] +email = ["email-validator (>=1.0.3)"] + +[[package]] +name = "pydocstyle" +version = "6.1.1" +description = "Python docstring style checker" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +snowballstemmer = "*" + +[package.extras] +toml = ["toml"] + +[[package]] +name = "pyflakes" +version = "2.2.0" +description = "passive checker of Python programs" +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[[package]] +name = "pygments" +version = "2.11.1" +description = "Pygments is a syntax highlighting package written in Python." +category = "main" +optional = false +python-versions = ">=3.5" + +[[package]] +name = "pylint" +version = "2.12.2" +description = "python code static checker" +category = "dev" +optional = false +python-versions = ">=3.6.2" + +[package.dependencies] +astroid = ">=2.9.0,<2.10" +colorama = {version = "*", markers = "sys_platform == \"win32\""} +isort = ">=4.2.5,<6" +mccabe = ">=0.6,<0.7" +platformdirs = ">=2.2.0" +toml = ">=0.9.2" +typing-extensions = {version = ">=3.10.0", markers = "python_version < \"3.10\""} + +[[package]] +name = "pyparsing" +version = "3.0.6" +description = "Python parsing module" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.extras] +diagrams = ["jinja2", "railroad-diagrams"] + +[[package]] +name = "pytest" +version = "6.0.1" +description = "pytest: simple powerful testing with Python" +category = "dev" +optional = false +python-versions = ">=3.5" + +[package.dependencies] +atomicwrites = {version = ">=1.0", markers = "sys_platform == \"win32\""} +attrs = ">=17.4.0" +colorama = {version = "*", markers = "sys_platform == \"win32\""} +iniconfig = "*" +more-itertools = ">=4.0.0" +packaging = "*" +pluggy = ">=0.12,<1.0" +py = ">=1.8.2" +toml = "*" + +[package.extras] +checkqa_mypy = ["mypy (==0.780)"] +testing = ["argcomplete", "hypothesis (>=3.56)", "mock", "nose", "requests", "xmlschema"] + +[[package]] +name = "pytest-runner" +version = "5.2" +description = "Invoke py.test as distutils command with dependency resolution" +category = "dev" +optional = false +python-versions = ">=2.7" + +[package.extras] +docs = ["sphinx", "jaraco.packaging (>=3.2)", "rst.linker (>=1.9)"] +testing = ["pytest (>=3.5,!=3.7.3)", "pytest-checkdocs", "pytest-flake8", "pytest-black-multipy", "pytest-cov", "pytest-virtualenv"] + +[[package]] +name = "python-dateutil" +version = "2.8.2" +description = "Extensions to the standard Python datetime module" +category = "main" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7" + +[package.dependencies] +six = ">=1.5" + +[[package]] +name = "pytz" +version = "2021.3" +description = "World timezone definitions, modern and historical" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "pyupgrade" +version = "2.31.0" +description = "A tool to automatically upgrade syntax for newer versions." +category = "dev" +optional = false +python-versions = ">=3.6.1" + +[package.dependencies] +tokenize-rt = ">=3.2.0" + +[[package]] +name = "pywin32-ctypes" +version = "0.2.0" +description = "" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "pyyaml" +version = "5.3.1" +description = "YAML parser and emitter for Python" +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" + +[[package]] +name = "readme-renderer" +version = "32.0" +description = "readme_renderer is a library for rendering \"readme\" descriptions for Warehouse" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +bleach = ">=2.1.0" +docutils = ">=0.13.1" +Pygments = ">=2.5.1" + +[package.extras] +md = ["cmarkgfm (>=0.5.0,<0.7.0)"] + +[[package]] +name = "requests" +version = "2.27.0" +description = "Python HTTP for Humans." +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*" + +[package.dependencies] +certifi = ">=2017.4.17" +charset-normalizer = {version = ">=2.0.0,<2.1.0", markers = "python_version >= \"3\""} +idna = {version = ">=2.5,<4", markers = "python_version >= \"3\""} +urllib3 = ">=1.21.1,<1.27" + +[package.extras] +socks = ["PySocks (>=1.5.6,!=1.5.7)", "win-inet-pton"] +use_chardet_on_py3 = ["chardet (>=3.0.2,<5)"] + +[[package]] +name = "requests-oauthlib" +version = "1.3.0" +description = "OAuthlib authentication support for Requests." +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + +[package.dependencies] +oauthlib = ">=3.0.0" +requests = ">=2.0.0" + +[package.extras] +rsa = ["oauthlib[signedtoken] (>=3.0.0)"] + +[[package]] +name = "requests-toolbelt" +version = "0.9.1" +description = "A utility belt for advanced users of python-requests" +category = "dev" +optional = false +python-versions = "*" + +[package.dependencies] +requests = ">=2.0.1,<3.0.0" + +[[package]] +name = "rfc3986" +version = "1.5.0" +description = "Validating URI References per RFC 3986" +category = "dev" +optional = false +python-versions = "*" + +[package.extras] +idna2008 = ["idna"] + +[[package]] +name = "rsa" +version = "4.8" +description = "Pure-Python RSA implementation" +category = "main" +optional = false +python-versions = ">=3.6,<4" + +[package.dependencies] +pyasn1 = ">=0.1.3" + +[[package]] +name = "safety" +version = "1.10.3" +description = "Checks installed dependencies for known vulnerabilities." +category = "dev" +optional = false +python-versions = ">=3.5" + +[package.dependencies] +Click = ">=6.0" +dparse = ">=0.5.1" +packaging = "*" +requests = "*" + +[[package]] +name = "scikit-learn" +version = "0.23.2" +description = "A set of python modules for machine learning and data mining" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +joblib = ">=0.11" +numpy = ">=1.13.3" +scipy = ">=0.19.1" +threadpoolctl = ">=2.0.0" + +[package.extras] +alldeps = ["numpy (>=1.13.3)", "scipy (>=0.19.1)"] + +[[package]] +name = "scipy" +version = "1.6.1" +description = "SciPy: Scientific Library for Python" +category = "main" +optional = false +python-versions = ">=3.7" + +[package.dependencies] +numpy = ">=1.16.5" + +[[package]] +name = "secretstorage" +version = "3.3.1" +description = "Python bindings to FreeDesktop.org Secret Service API" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +cryptography = ">=2.0" +jeepney = ">=0.6" + +[[package]] +name = "six" +version = "1.16.0" +description = "Python 2 and 3 compatibility utilities" +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*" + +[[package]] +name = "skl2onnx" +version = "1.10.3" +description = "Convert scikit-learn models to ONNX" +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +numpy = ">=1.15" +onnx = ">=1.2.1" +onnxconverter-common = ">=1.7.0" +protobuf = "*" +scikit-learn = ">=0.19" +scipy = ">=1.0" + +[[package]] +name = "smmap" +version = "5.0.0" +description = "A pure Python implementation of a sliding window memory map manager" +category = "dev" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "snowballstemmer" +version = "2.2.0" +description = "This package provides 29 stemmers for 28 languages generated from Snowball algorithms." +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "sphinx" +version = "3.2.1" +description = "Python documentation generator" +category = "dev" +optional = false +python-versions = ">=3.5" + +[package.dependencies] +alabaster = ">=0.7,<0.8" +babel = ">=1.3" +colorama = {version = ">=0.3.5", markers = "sys_platform == \"win32\""} +docutils = ">=0.12" +imagesize = "*" +Jinja2 = ">=2.3" +packaging = "*" +Pygments = ">=2.0" +requests = ">=2.5.0" +snowballstemmer = ">=1.1" +sphinxcontrib-applehelp = "*" +sphinxcontrib-devhelp = "*" +sphinxcontrib-htmlhelp = "*" +sphinxcontrib-jsmath = "*" +sphinxcontrib-qthelp = "*" +sphinxcontrib-serializinghtml = "*" + +[package.extras] +docs = ["sphinxcontrib-websupport"] +lint = ["flake8 (>=3.5.0)", "flake8-import-order", "mypy (>=0.780)", "docutils-stubs"] +test = ["pytest", "pytest-cov", "html5lib", "typed-ast", "cython"] + +[[package]] +name = "sphinxcontrib-applehelp" +version = "1.0.2" +description = "sphinxcontrib-applehelp is a sphinx extension which outputs Apple help books" +category = "dev" +optional = false +python-versions = ">=3.5" + +[package.extras] +lint = ["flake8", "mypy", "docutils-stubs"] +test = ["pytest"] + +[[package]] +name = "sphinxcontrib-devhelp" +version = "1.0.2" +description = "sphinxcontrib-devhelp is a sphinx extension which outputs Devhelp document." +category = "dev" +optional = false +python-versions = ">=3.5" + +[package.extras] +lint = ["flake8", "mypy", "docutils-stubs"] +test = ["pytest"] + +[[package]] +name = "sphinxcontrib-htmlhelp" +version = "2.0.0" +description = "sphinxcontrib-htmlhelp is a sphinx extension which renders HTML help files" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.extras] +lint = ["flake8", "mypy", "docutils-stubs"] +test = ["pytest", "html5lib"] + +[[package]] +name = "sphinxcontrib-jsmath" +version = "1.0.1" +description = "A sphinx extension which renders display math in HTML via JavaScript" +category = "dev" +optional = false +python-versions = ">=3.5" + +[package.extras] +test = ["pytest", "flake8", "mypy"] + +[[package]] +name = "sphinxcontrib-qthelp" +version = "1.0.3" +description = "sphinxcontrib-qthelp is a sphinx extension which outputs QtHelp document." +category = "dev" +optional = false +python-versions = ">=3.5" + +[package.extras] +lint = ["flake8", "mypy", "docutils-stubs"] +test = ["pytest"] + +[[package]] +name = "sphinxcontrib-serializinghtml" +version = "1.1.5" +description = "sphinxcontrib-serializinghtml is a sphinx extension which outputs \"serialized\" HTML files (json and pickle)." +category = "dev" +optional = false +python-versions = ">=3.5" + +[package.extras] +lint = ["flake8", "mypy", "docutils-stubs"] +test = ["pytest"] + +[[package]] +name = "starlette" +version = "0.14.2" +description = "The little ASGI library that shines." +category = "main" +optional = false +python-versions = ">=3.6" + +[package.extras] +full = ["aiofiles", "graphene", "itsdangerous", "jinja2", "python-multipart", "pyyaml", "requests"] + +[[package]] +name = "stevedore" +version = "3.5.0" +description = "Manage dynamic plugins for Python applications" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +pbr = ">=2.0.0,<2.1.0 || >2.1.0" + +[[package]] +name = "tensorboard" +version = "2.7.0" +description = "TensorBoard lets you watch Tensors Flow" +category = "main" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +absl-py = ">=0.4" +google-auth = ">=1.6.3,<3" +google-auth-oauthlib = ">=0.4.1,<0.5" +grpcio = ">=1.24.3" +markdown = ">=2.6.8" +numpy = ">=1.12.0" +protobuf = ">=3.6.0" +requests = ">=2.21.0,<3" +tensorboard-data-server = ">=0.6.0,<0.7.0" +tensorboard-plugin-wit = ">=1.6.0" +werkzeug = ">=0.11.15" + +[[package]] +name = "tensorboard-data-server" +version = "0.6.1" +description = "Fast data loading for TensorBoard" +category = "main" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "tensorboard-plugin-wit" +version = "1.8.0" +description = "What-If Tool TensorBoard plugin." +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "tensorflow" +version = "2.3.4" +description = "TensorFlow is an open source machine learning framework for everyone." +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +absl-py = ">=0.7.0" +astunparse = "1.6.3" +gast = "0.3.3" +google-pasta = ">=0.1.8" +grpcio = ">=1.8.6" +h5py = ">=2.10.0,<2.11.0" +keras-preprocessing = ">=1.1.1,<1.2" +numpy = ">=1.16.0,<1.19.0" +opt-einsum = ">=2.3.2" +protobuf = ">=3.9.2" +six = ">=1.12.0" +tensorboard = ">=2.3.0,<3" +tensorflow-estimator = ">=2.3.0,<2.4.0" +termcolor = ">=1.1.0" +wrapt = ">=1.11.1" + +[[package]] +name = "tensorflow-estimator" +version = "2.3.0" +description = "TensorFlow Estimator." +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "termcolor" +version = "1.1.0" +description = "ANSII Color formatting for output in terminal." +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "threadpoolctl" +version = "3.0.0" +description = "threadpoolctl" +category = "main" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "tokenize-rt" +version = "4.2.1" +description = "A wrapper around the stdlib `tokenize` which roundtrips." +category = "dev" +optional = false +python-versions = ">=3.6.1" + +[[package]] +name = "toml" +version = "0.10.2" +description = "Python Library for Tom's Obvious, Minimal Language" +category = "dev" +optional = false +python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*" + +[[package]] +name = "tomli" +version = "1.2.3" +description = "A lil' TOML parser" +category = "dev" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "tox" +version = "3.19.0" +description = "tox is a generic virtualenv management and test command line tool" +category = "dev" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7" + +[package.dependencies] +colorama = {version = ">=0.4.1", markers = "platform_system == \"Windows\""} +filelock = ">=3.0.0" +packaging = ">=14" +pluggy = ">=0.12.0" +py = ">=1.4.17" +six = ">=1.14.0" +toml = ">=0.9.4" +virtualenv = ">=16.0.0,<20.0.0 || >20.0.0,<20.0.1 || >20.0.1,<20.0.2 || >20.0.2,<20.0.3 || >20.0.3,<20.0.4 || >20.0.4,<20.0.5 || >20.0.5,<20.0.6 || >20.0.6,<20.0.7 || >20.0.7" + +[package.extras] +docs = ["pygments-github-lexers (>=0.0.5)", "sphinx (>=2.0.0)", "sphinxcontrib-autoprogram (>=0.1.5)", "towncrier (>=18.5.0)"] +testing = ["flaky (>=3.4.0)", "freezegun (>=0.3.11)", "pathlib2 (>=2.3.3)", "psutil (>=5.6.1)", "pytest (>=4.0.0)", "pytest-cov (>=2.5.1)", "pytest-mock (>=1.10.0)", "pytest-randomly (>=1.0.0)", "pytest-xdist (>=1.22.2)"] + +[[package]] +name = "tqdm" +version = "4.62.3" +description = "Fast, Extensible Progress Meter" +category = "dev" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7" + +[package.dependencies] +colorama = {version = "*", markers = "platform_system == \"Windows\""} + +[package.extras] +dev = ["py-make (>=0.1.0)", "twine", "wheel"] +notebook = ["ipywidgets (>=6)"] +telegram = ["requests"] + +[[package]] +name = "traitlets" +version = "5.1.1" +description = "Traitlets Python configuration system" +category = "main" +optional = false +python-versions = ">=3.7" + +[package.extras] +test = ["pytest"] + +[[package]] +name = "twine" +version = "3.2.0" +description = "Collection of utilities for publishing packages on PyPI" +category = "dev" +optional = false +python-versions = ">=3.6" + +[package.dependencies] +colorama = ">=0.4.3" +keyring = ">=15.1" +pkginfo = ">=1.4.2" +readme-renderer = ">=21.0" +requests = ">=2.20" +requests-toolbelt = ">=0.8.0,<0.9.0 || >0.9.0" +rfc3986 = ">=1.4.0" +tqdm = ">=4.14" + +[[package]] +name = "typing-extensions" +version = "4.0.1" +description = "Backported and Experimental Type Hints for Python 3.6+" +category = "main" +optional = false +python-versions = ">=3.6" + +[[package]] +name = "urllib3" +version = "1.26.7" +description = "HTTP library with thread-safe connection pooling, file post, and more." +category = "main" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <4" + +[package.extras] +brotli = ["brotlipy (>=0.6.0)"] +secure = ["pyOpenSSL (>=0.14)", "cryptography (>=1.3.4)", "idna (>=2.0.0)", "certifi", "ipaddress"] +socks = ["PySocks (>=1.5.6,!=1.5.7,<2.0)"] + +[[package]] +name = "uvicorn" +version = "0.14.0" +description = "The lightning-fast ASGI server." +category = "main" +optional = false +python-versions = "*" + +[package.dependencies] +asgiref = ">=3.3.4" +click = ">=7" +h11 = ">=0.8" + +[package.extras] +standard = ["websockets (>=9.1)", "httptools (>=0.2.0,<0.3.0)", "watchgod (>=0.6)", "python-dotenv (>=0.13)", "PyYAML (>=5.1)", "uvloop (>=0.14.0,!=0.15.0,!=0.15.1)", "colorama (>=0.4)"] + +[[package]] +name = "virtualenv" +version = "20.13.0" +description = "Virtual Python Environment builder" +category = "dev" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7" + +[package.dependencies] +distlib = ">=0.3.1,<1" +filelock = ">=3.2,<4" +platformdirs = ">=2,<3" +six = ">=1.9.0,<2" + +[package.extras] +docs = ["proselint (>=0.10.2)", "sphinx (>=3)", "sphinx-argparse (>=0.2.5)", "sphinx-rtd-theme (>=0.4.3)", "towncrier (>=21.3)"] +testing = ["coverage (>=4)", "coverage-enable-subprocess (>=1)", "flaky (>=3)", "pytest (>=4)", "pytest-env (>=0.6.2)", "pytest-freezegun (>=0.4.1)", "pytest-mock (>=2)", "pytest-randomly (>=1)", "pytest-timeout (>=1)", "packaging (>=20.0)"] + +[[package]] +name = "watchdog" +version = "0.10.3" +description = "Filesystem events monitoring" +category = "dev" +optional = false +python-versions = "*" + +[package.dependencies] +pathtools = ">=0.1.1" + +[package.extras] +watchmedo = ["PyYAML (>=3.10)", "argh (>=0.24.1)"] + +[[package]] +name = "wcwidth" +version = "0.2.5" +description = "Measures the displayed width of unicode strings in a terminal" +category = "main" +optional = false +python-versions = "*" + +[[package]] +name = "webencodings" +version = "0.5.1" +description = "Character encoding aliases for legacy web content" +category = "dev" +optional = false +python-versions = "*" + +[[package]] +name = "werkzeug" +version = "2.0.2" +description = "The comprehensive WSGI web application library." +category = "main" +optional = false +python-versions = ">=3.6" + +[package.extras] +watchdog = ["watchdog"] + +[[package]] +name = "wrapt" +version = "1.13.3" +description = "Module for decorators, wrappers and monkey patching." +category = "main" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7" + +[[package]] +name = "zipp" +version = "3.7.0" +description = "Backport of pathlib-compatible object wrapper for zip files" +category = "main" +optional = false +python-versions = ">=3.7" + +[package.extras] +docs = ["sphinx", "jaraco.packaging (>=8.2)", "rst.linker (>=1.9)"] +testing = ["pytest (>=6)", "pytest-checkdocs (>=2.4)", "pytest-flake8", "pytest-cov", "pytest-enabler (>=1.0.1)", "jaraco.itertools", "func-timeout", "pytest-black (>=0.3.7)", "pytest-mypy"] + +[metadata] +lock-version = "1.1" +python-versions = "^3.8" +content-hash = "d357dcec867400395eafaf5d399df3c315723c56fe1f6c86e0566167e1b582aa" + +[metadata.files] +absl-py = [ + {file = "absl-py-1.0.0.tar.gz", hash = "sha256:ac511215c01ee9ae47b19716599e8ccfa746f2e18de72bdf641b79b22afa27ea"}, + {file = "absl_py-1.0.0-py3-none-any.whl", hash = "sha256:84e6dcdc69c947d0c13e5457d056bd43cade4c2393dce00d684aedea77ddc2a3"}, +] +alabaster = [ + {file = "alabaster-0.7.12-py2.py3-none-any.whl", hash = "sha256:446438bdcca0e05bd45ea2de1668c1d9b032e1a9154c2c259092d77031ddd359"}, + {file = "alabaster-0.7.12.tar.gz", hash = "sha256:a661d72d58e6ea8a57f7a86e37d86716863ee5e92788398526d58b26a4e4dc02"}, +] +appnope = [ + {file = "appnope-0.1.2-py2.py3-none-any.whl", hash = "sha256:93aa393e9d6c54c5cd570ccadd8edad61ea0c4b9ea7a01409020c9aa019eb442"}, + {file = "appnope-0.1.2.tar.gz", hash = "sha256:dd83cd4b5b460958838f6eb3000c660b1f9caf2a5b1de4264e941512f603258a"}, +] +asgiref = [ + {file = "asgiref-3.4.1-py3-none-any.whl", hash = "sha256:ffc141aa908e6f175673e7b1b3b7af4fdb0ecb738fc5c8b88f69f055c2415214"}, + {file = "asgiref-3.4.1.tar.gz", hash = "sha256:4ef1ab46b484e3c706329cedeff284a5d40824200638503f5768edb6de7d58e9"}, +] +astroid = [ + {file = "astroid-2.9.2-py3-none-any.whl", hash = "sha256:aa296702f1a5c3102c860de49473aaa90a7f6d221555d5cf2678940a9be32a4e"}, + {file = "astroid-2.9.2.tar.gz", hash = "sha256:72ace9c3333e274e9248168fc4f3e300da8545af1c303bd69197027f49e2bfff"}, +] +astunparse = [ + {file = "astunparse-1.6.3-py2.py3-none-any.whl", hash = "sha256:c2652417f2c8b5bb325c885ae329bdf3f86424075c4fd1a128674bc6fba4b8e8"}, + {file = "astunparse-1.6.3.tar.gz", hash = "sha256:5ad93a8456f0d084c3456d059fd9a92cce667963232cbf763eac3bc5b7940872"}, +] +atomicwrites = [ + {file = "atomicwrites-1.4.0-py2.py3-none-any.whl", hash = "sha256:6d1784dea7c0c8d4a5172b6c620f40b6e4cbfdf96d783691f2e1302a7b88e197"}, + {file = "atomicwrites-1.4.0.tar.gz", hash = "sha256:ae70396ad1a434f9c7046fd2dd196fc04b12f9e91ffb859164193be8b6168a7a"}, +] +attrs = [ + {file = "attrs-21.4.0-py2.py3-none-any.whl", hash = "sha256:2d27e3784d7a565d36ab851fe94887c5eccd6a463168875832a1be79c82828b4"}, + {file = "attrs-21.4.0.tar.gz", hash = "sha256:626ba8234211db98e869df76230a137c4c40a12d72445c45d5f5b716f076e2fd"}, +] +autokeras = [ + {file = "autokeras-1.0.16.post1-py3-none-any.whl", hash = "sha256:945fabb65180efbded426ec3873e46c852baa08984e3fb145c1c040524140b41"}, + {file = "autokeras-1.0.16.post1.tar.gz", hash = "sha256:f1bd559fe4e75734dcd3c4ed5552b04652764b9afe262354f0cdecb2f6686199"}, +] +babel = [ + {file = "Babel-2.9.1-py2.py3-none-any.whl", hash = "sha256:ab49e12b91d937cd11f0b67cb259a57ab4ad2b59ac7a3b41d6c06c0ac5b0def9"}, + {file = "Babel-2.9.1.tar.gz", hash = "sha256:bc0c176f9f6a994582230df350aa6e05ba2ebe4b3ac317eab29d9be5d2768da0"}, +] +backcall = [ + {file = "backcall-0.2.0-py2.py3-none-any.whl", hash = "sha256:fbbce6a29f263178a1f7915c1940bde0ec2b2a967566fe1c65c1dfb7422bd255"}, + {file = "backcall-0.2.0.tar.gz", hash = "sha256:5cbdbf27be5e7cfadb448baf0aa95508f91f2bbc6c6437cd9cd06e2a4c215e1e"}, +] +bandit = [ + {file = "bandit-1.7.1-py3-none-any.whl", hash = "sha256:f5acd838e59c038a159b5c621cf0f8270b279e884eadd7b782d7491c02add0d4"}, + {file = "bandit-1.7.1.tar.gz", hash = "sha256:a81b00b5436e6880fa8ad6799bc830e02032047713cbb143a12939ac67eb756c"}, +] +black = [ + {file = "black-21.12b0-py3-none-any.whl", hash = "sha256:a615e69ae185e08fdd73e4715e260e2479c861b5740057fde6e8b4e3b7dd589f"}, + {file = "black-21.12b0.tar.gz", hash = "sha256:77b80f693a569e2e527958459634f18df9b0ba2625ba4e0c2d5da5be42e6f2b3"}, +] +bleach = [ + {file = "bleach-4.1.0-py2.py3-none-any.whl", hash = "sha256:4d2651ab93271d1129ac9cbc679f524565cc8a1b791909c4a51eac4446a15994"}, + {file = "bleach-4.1.0.tar.gz", hash = "sha256:0900d8b37eba61a802ee40ac0061f8c2b5dee29c1927dd1d233e075ebf5a71da"}, +] +bump2version = [ + {file = "bump2version-1.0.0-py2.py3-none-any.whl", hash = "sha256:477f0e18a0d58e50bb3dbc9af7fcda464fd0ebfc7a6151d8888602d7153171a0"}, + {file = "bump2version-1.0.0.tar.gz", hash = "sha256:cd4f3a231305e405ed8944d8ff35bd742d9bc740ad62f483bd0ca21ce7131984"}, +] +cachetools = [ + {file = "cachetools-4.2.4-py3-none-any.whl", hash = "sha256:92971d3cb7d2a97efff7c7bb1657f21a8f5fb309a37530537c71b1774189f2d1"}, + {file = "cachetools-4.2.4.tar.gz", hash = "sha256:89ea6f1b638d5a73a4f9226be57ac5e4f399d22770b92355f92dcb0f7f001693"}, +] +certifi = [ + {file = "certifi-2021.10.8-py2.py3-none-any.whl", hash = "sha256:d62a0163eb4c2344ac042ab2bdf75399a71a2d8c7d47eac2e2ee91b9d6339569"}, + {file = "certifi-2021.10.8.tar.gz", hash = "sha256:78884e7c1d4b00ce3cea67b44566851c4343c120abd683433ce934a68ea58872"}, +] +cffi = [ + {file = "cffi-1.15.0-cp27-cp27m-macosx_10_9_x86_64.whl", hash = "sha256:c2502a1a03b6312837279c8c1bd3ebedf6c12c4228ddbad40912d671ccc8a962"}, + {file = "cffi-1.15.0-cp27-cp27m-manylinux1_i686.whl", hash = "sha256:23cfe892bd5dd8941608f93348c0737e369e51c100d03718f108bf1add7bd6d0"}, + {file = "cffi-1.15.0-cp27-cp27m-manylinux1_x86_64.whl", hash = "sha256:41d45de54cd277a7878919867c0f08b0cf817605e4eb94093e7516505d3c8d14"}, + {file = "cffi-1.15.0-cp27-cp27m-win32.whl", hash = "sha256:4a306fa632e8f0928956a41fa8e1d6243c71e7eb59ffbd165fc0b41e316b2474"}, + {file = "cffi-1.15.0-cp27-cp27m-win_amd64.whl", hash = "sha256:e7022a66d9b55e93e1a845d8c9eba2a1bebd4966cd8bfc25d9cd07d515b33fa6"}, + {file = "cffi-1.15.0-cp27-cp27mu-manylinux1_i686.whl", hash = "sha256:14cd121ea63ecdae71efa69c15c5543a4b5fbcd0bbe2aad864baca0063cecf27"}, + {file = "cffi-1.15.0-cp27-cp27mu-manylinux1_x86_64.whl", hash = "sha256:d4d692a89c5cf08a8557fdeb329b82e7bf609aadfaed6c0d79f5a449a3c7c023"}, + {file = "cffi-1.15.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:0104fb5ae2391d46a4cb082abdd5c69ea4eab79d8d44eaaf79f1b1fd806ee4c2"}, + {file = "cffi-1.15.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:91ec59c33514b7c7559a6acda53bbfe1b283949c34fe7440bcf917f96ac0723e"}, + {file = "cffi-1.15.0-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:f5c7150ad32ba43a07c4479f40241756145a1f03b43480e058cfd862bf5041c7"}, + {file = "cffi-1.15.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:00c878c90cb53ccfaae6b8bc18ad05d2036553e6d9d1d9dbcf323bbe83854ca3"}, + {file = "cffi-1.15.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:abb9a20a72ac4e0fdb50dae135ba5e77880518e742077ced47eb1499e29a443c"}, + {file = "cffi-1.15.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:a5263e363c27b653a90078143adb3d076c1a748ec9ecc78ea2fb916f9b861962"}, + {file = "cffi-1.15.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:f54a64f8b0c8ff0b64d18aa76675262e1700f3995182267998c31ae974fbc382"}, + {file = "cffi-1.15.0-cp310-cp310-win32.whl", hash = "sha256:c21c9e3896c23007803a875460fb786118f0cdd4434359577ea25eb556e34c55"}, + {file = "cffi-1.15.0-cp310-cp310-win_amd64.whl", hash = "sha256:5e069f72d497312b24fcc02073d70cb989045d1c91cbd53979366077959933e0"}, + {file = "cffi-1.15.0-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:64d4ec9f448dfe041705426000cc13e34e6e5bb13736e9fd62e34a0b0c41566e"}, + {file = "cffi-1.15.0-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2756c88cbb94231c7a147402476be2c4df2f6078099a6f4a480d239a8817ae39"}, + {file = "cffi-1.15.0-cp36-cp36m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3b96a311ac60a3f6be21d2572e46ce67f09abcf4d09344c49274eb9e0bf345fc"}, + {file = "cffi-1.15.0-cp36-cp36m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:75e4024375654472cc27e91cbe9eaa08567f7fbdf822638be2814ce059f58032"}, + {file = "cffi-1.15.0-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:59888172256cac5629e60e72e86598027aca6bf01fa2465bdb676d37636573e8"}, + {file = "cffi-1.15.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:27c219baf94952ae9d50ec19651a687b826792055353d07648a5695413e0c605"}, + {file = "cffi-1.15.0-cp36-cp36m-win32.whl", hash = "sha256:4958391dbd6249d7ad855b9ca88fae690783a6be9e86df65865058ed81fc860e"}, + {file = "cffi-1.15.0-cp36-cp36m-win_amd64.whl", hash = "sha256:f6f824dc3bce0edab5f427efcfb1d63ee75b6fcb7282900ccaf925be84efb0fc"}, + {file = "cffi-1.15.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:06c48159c1abed75c2e721b1715c379fa3200c7784271b3c46df01383b593636"}, + {file = "cffi-1.15.0-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:c2051981a968d7de9dd2d7b87bcb9c939c74a34626a6e2f8181455dd49ed69e4"}, + {file = "cffi-1.15.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:fd8a250edc26254fe5b33be00402e6d287f562b6a5b2152dec302fa15bb3e997"}, + {file = "cffi-1.15.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:91d77d2a782be4274da750752bb1650a97bfd8f291022b379bb8e01c66b4e96b"}, + {file = "cffi-1.15.0-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:45db3a33139e9c8f7c09234b5784a5e33d31fd6907800b316decad50af323ff2"}, + {file = "cffi-1.15.0-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:263cc3d821c4ab2213cbe8cd8b355a7f72a8324577dc865ef98487c1aeee2bc7"}, + {file = "cffi-1.15.0-cp37-cp37m-win32.whl", hash = "sha256:17771976e82e9f94976180f76468546834d22a7cc404b17c22df2a2c81db0c66"}, + {file = "cffi-1.15.0-cp37-cp37m-win_amd64.whl", hash = "sha256:3415c89f9204ee60cd09b235810be700e993e343a408693e80ce7f6a40108029"}, + {file = "cffi-1.15.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:4238e6dab5d6a8ba812de994bbb0a79bddbdf80994e4ce802b6f6f3142fcc880"}, + {file = "cffi-1.15.0-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:0808014eb713677ec1292301ea4c81ad277b6cdf2fdd90fd540af98c0b101d20"}, + {file = "cffi-1.15.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:57e9ac9ccc3101fac9d6014fba037473e4358ef4e89f8e181f8951a2c0162024"}, + {file = "cffi-1.15.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8b6c2ea03845c9f501ed1313e78de148cd3f6cad741a75d43a29b43da27f2e1e"}, + {file = "cffi-1.15.0-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:10dffb601ccfb65262a27233ac273d552ddc4d8ae1bf93b21c94b8511bffe728"}, + {file = "cffi-1.15.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:786902fb9ba7433aae840e0ed609f45c7bcd4e225ebb9c753aa39725bb3e6ad6"}, + {file = "cffi-1.15.0-cp38-cp38-win32.whl", hash = "sha256:da5db4e883f1ce37f55c667e5c0de439df76ac4cb55964655906306918e7363c"}, + {file = "cffi-1.15.0-cp38-cp38-win_amd64.whl", hash = "sha256:181dee03b1170ff1969489acf1c26533710231c58f95534e3edac87fff06c443"}, + {file = "cffi-1.15.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:45e8636704eacc432a206ac7345a5d3d2c62d95a507ec70d62f23cd91770482a"}, + {file = "cffi-1.15.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:31fb708d9d7c3f49a60f04cf5b119aeefe5644daba1cd2a0fe389b674fd1de37"}, + {file = "cffi-1.15.0-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:6dc2737a3674b3e344847c8686cf29e500584ccad76204efea14f451d4cc669a"}, + {file = "cffi-1.15.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:74fdfdbfdc48d3f47148976f49fab3251e550a8720bebc99bf1483f5bfb5db3e"}, + {file = "cffi-1.15.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ffaa5c925128e29efbde7301d8ecaf35c8c60ffbcd6a1ffd3a552177c8e5e796"}, + {file = "cffi-1.15.0-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3f7d084648d77af029acb79a0ff49a0ad7e9d09057a9bf46596dac9514dc07df"}, + {file = "cffi-1.15.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:ef1f279350da2c586a69d32fc8733092fd32cc8ac95139a00377841f59a3f8d8"}, + {file = "cffi-1.15.0-cp39-cp39-win32.whl", hash = "sha256:2a23af14f408d53d5e6cd4e3d9a24ff9e05906ad574822a10563efcef137979a"}, + {file = "cffi-1.15.0-cp39-cp39-win_amd64.whl", hash = "sha256:3773c4d81e6e818df2efbc7dd77325ca0dcb688116050fb2b3011218eda36139"}, + {file = "cffi-1.15.0.tar.gz", hash = "sha256:920f0d66a896c2d99f0adbb391f990a84091179542c205fa53ce5787aff87954"}, +] +cfgv = [ + {file = "cfgv-3.3.1-py2.py3-none-any.whl", hash = "sha256:c6a0883f3917a037485059700b9e75da2464e6c27051014ad85ba6aaa5884426"}, + {file = "cfgv-3.3.1.tar.gz", hash = "sha256:f5a830efb9ce7a445376bb66ec94c638a9787422f96264c98edc6bdeed8ab736"}, +] +charset-normalizer = [ + {file = "charset-normalizer-2.0.9.tar.gz", hash = "sha256:b0b883e8e874edfdece9c28f314e3dd5badf067342e42fb162203335ae61aa2c"}, + {file = "charset_normalizer-2.0.9-py3-none-any.whl", hash = "sha256:1eecaa09422db5be9e29d7fc65664e6c33bd06f9ced7838578ba40d58bdf3721"}, +] +click = [ + {file = "click-8.0.3-py3-none-any.whl", hash = "sha256:353f466495adaeb40b6b5f592f9f91cb22372351c84caeb068132442a4518ef3"}, + {file = "click-8.0.3.tar.gz", hash = "sha256:410e932b050f5eed773c4cda94de75971c89cdb3155a72a0831139a79e5ecb5b"}, +] +colorama = [ + {file = "colorama-0.4.4-py2.py3-none-any.whl", hash = "sha256:9f47eda37229f68eee03b24b9748937c7dc3868f906e8ba69fbcbdd3bc5dc3e2"}, + {file = "colorama-0.4.4.tar.gz", hash = "sha256:5941b2b48a20143d2267e95b1c2a7603ce057ee39fd88e7329b0c292aa16869b"}, +] +coverage = [ + {file = "coverage-5.2.1-cp27-cp27m-macosx_10_13_intel.whl", hash = "sha256:40f70f81be4d34f8d491e55936904db5c527b0711b2a46513641a5729783c2e4"}, + {file = "coverage-5.2.1-cp27-cp27m-macosx_10_9_x86_64.whl", hash = "sha256:675192fca634f0df69af3493a48224f211f8db4e84452b08d5fcebb9167adb01"}, + {file = "coverage-5.2.1-cp27-cp27m-manylinux1_i686.whl", hash = "sha256:2fcc8b58953d74d199a1a4d633df8146f0ac36c4e720b4a1997e9b6327af43a8"}, + {file = "coverage-5.2.1-cp27-cp27m-manylinux1_x86_64.whl", hash = "sha256:64c4f340338c68c463f1b56e3f2f0423f7b17ba6c3febae80b81f0e093077f59"}, + {file = "coverage-5.2.1-cp27-cp27m-win32.whl", hash = "sha256:52f185ffd3291196dc1aae506b42e178a592b0b60a8610b108e6ad892cfc1bb3"}, + {file = "coverage-5.2.1-cp27-cp27m-win_amd64.whl", hash = "sha256:30bc103587e0d3df9e52cd9da1dd915265a22fad0b72afe54daf840c984b564f"}, + {file = "coverage-5.2.1-cp27-cp27mu-manylinux1_i686.whl", hash = "sha256:9ea749fd447ce7fb1ac71f7616371f04054d969d412d37611716721931e36efd"}, + {file = "coverage-5.2.1-cp27-cp27mu-manylinux1_x86_64.whl", hash = "sha256:ce7866f29d3025b5b34c2e944e66ebef0d92e4a4f2463f7266daa03a1332a651"}, + {file = "coverage-5.2.1-cp35-cp35m-macosx_10_13_x86_64.whl", hash = "sha256:4869ab1c1ed33953bb2433ce7b894a28d724b7aa76c19b11e2878034a4e4680b"}, + {file = "coverage-5.2.1-cp35-cp35m-manylinux1_i686.whl", hash = "sha256:a3ee9c793ffefe2944d3a2bd928a0e436cd0ac2d9e3723152d6fd5398838ce7d"}, + {file = "coverage-5.2.1-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:28f42dc5172ebdc32622a2c3f7ead1b836cdbf253569ae5673f499e35db0bac3"}, + {file = "coverage-5.2.1-cp35-cp35m-win32.whl", hash = "sha256:e26c993bd4b220429d4ec8c1468eca445a4064a61c74ca08da7429af9bc53bb0"}, + {file = "coverage-5.2.1-cp35-cp35m-win_amd64.whl", hash = "sha256:4186fc95c9febeab5681bc3248553d5ec8c2999b8424d4fc3a39c9cba5796962"}, + {file = "coverage-5.2.1-cp36-cp36m-macosx_10_13_x86_64.whl", hash = "sha256:b360d8fd88d2bad01cb953d81fd2edd4be539df7bfec41e8753fe9f4456a5082"}, + {file = "coverage-5.2.1-cp36-cp36m-manylinux1_i686.whl", hash = "sha256:1adb6be0dcef0cf9434619d3b892772fdb48e793300f9d762e480e043bd8e716"}, + {file = "coverage-5.2.1-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:098a703d913be6fbd146a8c50cc76513d726b022d170e5e98dc56d958fd592fb"}, + {file = "coverage-5.2.1-cp36-cp36m-win32.whl", hash = "sha256:962c44070c281d86398aeb8f64e1bf37816a4dfc6f4c0f114756b14fc575621d"}, + {file = "coverage-5.2.1-cp36-cp36m-win_amd64.whl", hash = "sha256:b1ed2bdb27b4c9fc87058a1cb751c4df8752002143ed393899edb82b131e0546"}, + {file = "coverage-5.2.1-cp37-cp37m-macosx_10_13_x86_64.whl", hash = "sha256:c890728a93fffd0407d7d37c1e6083ff3f9f211c83b4316fae3778417eab9811"}, + {file = "coverage-5.2.1-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:538f2fd5eb64366f37c97fdb3077d665fa946d2b6d95447622292f38407f9258"}, + {file = "coverage-5.2.1-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:27ca5a2bc04d68f0776f2cdcb8bbd508bbe430a7bf9c02315cd05fb1d86d0034"}, + {file = "coverage-5.2.1-cp37-cp37m-win32.whl", hash = "sha256:aab75d99f3f2874733946a7648ce87a50019eb90baef931698f96b76b6769a46"}, + {file = "coverage-5.2.1-cp37-cp37m-win_amd64.whl", hash = "sha256:c2ff24df02a125b7b346c4c9078c8936da06964cc2d276292c357d64378158f8"}, + {file = "coverage-5.2.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:304fbe451698373dc6653772c72c5d5e883a4aadaf20343592a7abb2e643dae0"}, + {file = "coverage-5.2.1-cp38-cp38-manylinux1_i686.whl", hash = "sha256:c96472b8ca5dc135fb0aa62f79b033f02aa434fb03a8b190600a5ae4102df1fd"}, + {file = "coverage-5.2.1-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:8505e614c983834239f865da2dd336dcf9d72776b951d5dfa5ac36b987726e1b"}, + {file = "coverage-5.2.1-cp38-cp38-win32.whl", hash = "sha256:700997b77cfab016533b3e7dbc03b71d33ee4df1d79f2463a318ca0263fc29dd"}, + {file = "coverage-5.2.1-cp38-cp38-win_amd64.whl", hash = "sha256:46794c815e56f1431c66d81943fa90721bb858375fb36e5903697d5eef88627d"}, + {file = "coverage-5.2.1-cp39-cp39-macosx_10_13_x86_64.whl", hash = "sha256:16042dc7f8e632e0dcd5206a5095ebd18cb1d005f4c89694f7f8aafd96dd43a3"}, + {file = "coverage-5.2.1-cp39-cp39-manylinux1_i686.whl", hash = "sha256:c1bbb628ed5192124889b51204de27c575b3ffc05a5a91307e7640eff1d48da4"}, + {file = "coverage-5.2.1-cp39-cp39-manylinux1_x86_64.whl", hash = "sha256:4f6428b55d2916a69f8d6453e48a505c07b2245653b0aa9f0dee38785939f5e4"}, + {file = "coverage-5.2.1-cp39-cp39-win32.whl", hash = "sha256:9e536783a5acee79a9b308be97d3952b662748c4037b6a24cbb339dc7ed8eb89"}, + {file = "coverage-5.2.1-cp39-cp39-win_amd64.whl", hash = "sha256:b8f58c7db64d8f27078cbf2a4391af6aa4e4767cc08b37555c4ae064b8558d9b"}, + {file = "coverage-5.2.1.tar.gz", hash = "sha256:a34cb28e0747ea15e82d13e14de606747e9e484fb28d63c999483f5d5188e89b"}, +] +cryptography = [ + {file = "cryptography-36.0.1-cp36-abi3-macosx_10_10_universal2.whl", hash = "sha256:73bc2d3f2444bcfeac67dd130ff2ea598ea5f20b40e36d19821b4df8c9c5037b"}, + {file = "cryptography-36.0.1-cp36-abi3-macosx_10_10_x86_64.whl", hash = "sha256:2d87cdcb378d3cfed944dac30596da1968f88fb96d7fc34fdae30a99054b2e31"}, + {file = "cryptography-36.0.1-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:74d6c7e80609c0f4c2434b97b80c7f8fdfaa072ca4baab7e239a15d6d70ed73a"}, + {file = "cryptography-36.0.1-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl", hash = "sha256:6c0c021f35b421ebf5976abf2daacc47e235f8b6082d3396a2fe3ccd537ab173"}, + {file = "cryptography-36.0.1-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5d59a9d55027a8b88fd9fd2826c4392bd487d74bf628bb9d39beecc62a644c12"}, + {file = "cryptography-36.0.1-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0a817b961b46894c5ca8a66b599c745b9a3d9f822725221f0e0fe49dc043a3a3"}, + {file = "cryptography-36.0.1-cp36-abi3-manylinux_2_24_x86_64.whl", hash = "sha256:94ae132f0e40fe48f310bba63f477f14a43116f05ddb69d6fa31e93f05848ae2"}, + {file = "cryptography-36.0.1-cp36-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:7be0eec337359c155df191d6ae00a5e8bbb63933883f4f5dffc439dac5348c3f"}, + {file = "cryptography-36.0.1-cp36-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:e0344c14c9cb89e76eb6a060e67980c9e35b3f36691e15e1b7a9e58a0a6c6dc3"}, + {file = "cryptography-36.0.1-cp36-abi3-win32.whl", hash = "sha256:4caa4b893d8fad33cf1964d3e51842cd78ba87401ab1d2e44556826df849a8ca"}, + {file = "cryptography-36.0.1-cp36-abi3-win_amd64.whl", hash = "sha256:391432971a66cfaf94b21c24ab465a4cc3e8bf4a939c1ca5c3e3a6e0abebdbcf"}, + {file = "cryptography-36.0.1-pp37-pypy37_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:bb5829d027ff82aa872d76158919045a7c1e91fbf241aec32cb07956e9ebd3c9"}, + {file = "cryptography-36.0.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ebc15b1c22e55c4d5566e3ca4db8689470a0ca2babef8e3a9ee057a8b82ce4b1"}, + {file = "cryptography-36.0.1-pp37-pypy37_pp73-manylinux_2_24_x86_64.whl", hash = "sha256:596f3cd67e1b950bc372c33f1a28a0692080625592ea6392987dba7f09f17a94"}, + {file = "cryptography-36.0.1-pp38-pypy38_pp73-macosx_10_10_x86_64.whl", hash = "sha256:30ee1eb3ebe1644d1c3f183d115a8c04e4e603ed6ce8e394ed39eea4a98469ac"}, + {file = "cryptography-36.0.1-pp38-pypy38_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:ec63da4e7e4a5f924b90af42eddf20b698a70e58d86a72d943857c4c6045b3ee"}, + {file = "cryptography-36.0.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ca238ceb7ba0bdf6ce88c1b74a87bffcee5afbfa1e41e173b1ceb095b39add46"}, + {file = "cryptography-36.0.1-pp38-pypy38_pp73-manylinux_2_24_x86_64.whl", hash = "sha256:ca28641954f767f9822c24e927ad894d45d5a1e501767599647259cbf030b903"}, + {file = "cryptography-36.0.1-pp38-pypy38_pp73-win_amd64.whl", hash = "sha256:39bdf8e70eee6b1c7b289ec6e5d84d49a6bfa11f8b8646b5b3dfe41219153316"}, + {file = "cryptography-36.0.1.tar.gz", hash = "sha256:53e5c1dc3d7a953de055d77bef2ff607ceef7a2aac0353b5d630ab67f7423638"}, +] +darglint = [ + {file = "darglint-1.8.1-py3-none-any.whl", hash = "sha256:5ae11c259c17b0701618a20c3da343a3eb98b3bc4b5a83d31cdd94f5ebdced8d"}, + {file = "darglint-1.8.1.tar.gz", hash = "sha256:080d5106df149b199822e7ee7deb9c012b49891538f14a11be681044f0bb20da"}, +] +decorator = [ + {file = "decorator-5.1.0-py3-none-any.whl", hash = "sha256:7b12e7c3c6ab203a29e157335e9122cb03de9ab7264b137594103fd4a683b374"}, + {file = "decorator-5.1.0.tar.gz", hash = "sha256:e59913af105b9860aa2c8d3272d9de5a56a4e608db9a2f167a8480b323d529a7"}, +] +distlib = [ + {file = "distlib-0.3.4-py2.py3-none-any.whl", hash = "sha256:6564fe0a8f51e734df6333d08b8b94d4ea8ee6b99b5ed50613f731fd4089f34b"}, + {file = "distlib-0.3.4.zip", hash = "sha256:e4b58818180336dc9c529bfb9a0b58728ffc09ad92027a3f30b7cd91e3458579"}, +] +docutils = [ + {file = "docutils-0.18.1-py2.py3-none-any.whl", hash = "sha256:23010f129180089fbcd3bc08cfefccb3b890b0050e1ca00c867036e9d161b98c"}, + {file = "docutils-0.18.1.tar.gz", hash = "sha256:679987caf361a7539d76e584cbeddc311e3aee937877c87346f31debc63e9d06"}, +] +dparse = [ + {file = "dparse-0.5.1-py3-none-any.whl", hash = "sha256:e953a25e44ebb60a5c6efc2add4420c177f1d8404509da88da9729202f306994"}, + {file = "dparse-0.5.1.tar.gz", hash = "sha256:a1b5f169102e1c894f9a7d5ccf6f9402a836a5d24be80a986c7ce9eaed78f367"}, +] +fastapi = [ + {file = "fastapi-0.65.3-py3-none-any.whl", hash = "sha256:d3e3c0ac35110efb22ee3ed28201cf32f9d11a9a0e52d7dd676cad25f5219523"}, + {file = "fastapi-0.65.3.tar.gz", hash = "sha256:6ea2286e439c4ced7cce2b2862c25859601bf327a515c12dd6e431ef5d49d12f"}, +] +filelock = [ + {file = "filelock-3.4.2-py3-none-any.whl", hash = "sha256:cf0fc6a2f8d26bd900f19bf33915ca70ba4dd8c56903eeb14e1e7a2fd7590146"}, + {file = "filelock-3.4.2.tar.gz", hash = "sha256:38b4f4c989f9d06d44524df1b24bd19e167d851f19b50bf3e3559952dddc5b80"}, +] +flake8 = [ + {file = "flake8-3.8.3-py2.py3-none-any.whl", hash = "sha256:15e351d19611c887e482fb960eae4d44845013cc142d42896e9862f775d8cf5c"}, + {file = "flake8-3.8.3.tar.gz", hash = "sha256:f04b9fcbac03b0a3e58c0ab3a0ecc462e023a9faf046d57794184028123aa208"}, +] +gast = [ + {file = "gast-0.3.3-py2.py3-none-any.whl", hash = "sha256:8f46f5be57ae6889a4e16e2ca113b1703ef17f2b0abceb83793eaba9e1351a45"}, + {file = "gast-0.3.3.tar.gz", hash = "sha256:b881ef288a49aa81440d2c5eb8aeefd4c2bb8993d5f50edae7413a85bfdb3b57"}, +] +gitdb = [ + {file = "gitdb-4.0.9-py3-none-any.whl", hash = "sha256:8033ad4e853066ba6ca92050b9df2f89301b8fc8bf7e9324d412a63f8bf1a8fd"}, + {file = "gitdb-4.0.9.tar.gz", hash = "sha256:bac2fd45c0a1c9cf619e63a90d62bdc63892ef92387424b855792a6cabe789aa"}, +] +gitpython = [ + {file = "GitPython-3.1.24-py3-none-any.whl", hash = "sha256:dc0a7f2f697657acc8d7f89033e8b1ea94dd90356b2983bca89dc8d2ab3cc647"}, + {file = "GitPython-3.1.24.tar.gz", hash = "sha256:df83fdf5e684fef7c6ee2c02fc68a5ceb7e7e759d08b694088d0cacb4eba59e5"}, +] +google-auth = [ + {file = "google-auth-2.3.3.tar.gz", hash = "sha256:d83570a664c10b97a1dc6f8df87e5fdfff012f48f62be131e449c20dfc32630e"}, + {file = "google_auth-2.3.3-py2.py3-none-any.whl", hash = "sha256:a348a50b027679cb7dae98043ac8dbcc1d7951f06d8387496071a1e05a2465c0"}, +] +google-auth-oauthlib = [ + {file = "google-auth-oauthlib-0.4.6.tar.gz", hash = "sha256:a90a072f6993f2c327067bf65270046384cda5a8ecb20b94ea9a687f1f233a7a"}, + {file = "google_auth_oauthlib-0.4.6-py2.py3-none-any.whl", hash = "sha256:3f2a6e802eebbb6fb736a370fbf3b055edcb6b52878bf2f26330b5e041316c73"}, +] +google-pasta = [ + {file = "google-pasta-0.2.0.tar.gz", hash = "sha256:c9f2c8dfc8f96d0d5808299920721be30c9eec37f2389f28904f454565c8a16e"}, + {file = "google_pasta-0.2.0-py2-none-any.whl", hash = "sha256:4612951da876b1a10fe3960d7226f0c7682cf901e16ac06e473b267a5afa8954"}, + {file = "google_pasta-0.2.0-py3-none-any.whl", hash = "sha256:b32482794a366b5366a32c92a9a9201b107821889935a02b3e51f6b432ea84ed"}, +] +grpcio = [ + {file = "grpcio-1.43.0-cp310-cp310-linux_armv7l.whl", hash = "sha256:a4e786a8ee8b30b25d70ee52cda6d1dbba2a8ca2f1208d8e20ed8280774f15c8"}, + {file = "grpcio-1.43.0-cp310-cp310-macosx_10_10_universal2.whl", hash = "sha256:af9c3742f6c13575c0d4147a8454da0ff5308c4d9469462ff18402c6416942fe"}, + {file = "grpcio-1.43.0-cp310-cp310-manylinux_2_17_aarch64.whl", hash = "sha256:fdac966699707b5554b815acc272d81e619dd0999f187cd52a61aef075f870ee"}, + {file = "grpcio-1.43.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6e463b4aa0a6b31cf2e57c4abc1a1b53531a18a570baeed39d8d7b65deb16b7e"}, + {file = "grpcio-1.43.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f11d05402e0ac3a284443d8a432d3dfc76a6bd3f7b5858cddd75617af2d7bd9b"}, + {file = "grpcio-1.43.0-cp310-cp310-win32.whl", hash = "sha256:c36f418c925a41fccada8f7ae9a3d3e227bfa837ddbfddd3d8b0ac252d12dda9"}, + {file = "grpcio-1.43.0-cp310-cp310-win_amd64.whl", hash = "sha256:772b943f34374744f70236bbbe0afe413ed80f9ae6303503f85e2b421d4bca92"}, + {file = "grpcio-1.43.0-cp36-cp36m-linux_armv7l.whl", hash = "sha256:cbc9b83211d905859dcf234ad39d7193ff0f05bfc3269c364fb0d114ee71de59"}, + {file = "grpcio-1.43.0-cp36-cp36m-macosx_10_10_x86_64.whl", hash = "sha256:fb7229fa2a201a0c377ff3283174ec966da8f9fd7ffcc9a92f162d2e7fc9025b"}, + {file = "grpcio-1.43.0-cp36-cp36m-manylinux2010_i686.whl", hash = "sha256:17b75f220ee6923338155b4fcef4c38802b9a57bc57d112c9599a13a03e99f8d"}, + {file = "grpcio-1.43.0-cp36-cp36m-manylinux2010_x86_64.whl", hash = "sha256:6620a5b751b099b3b25553cfc03dfcd873cda06f9bb2ff7e9948ac7090e20f05"}, + {file = "grpcio-1.43.0-cp36-cp36m-manylinux_2_17_aarch64.whl", hash = "sha256:1898f999383baac5fcdbdef8ea5b1ef204f38dc211014eb6977ac6e55944d738"}, + {file = "grpcio-1.43.0-cp36-cp36m-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:47b6821238d8978014d23b1132713dac6c2d72cbb561cf257608b1673894f90a"}, + {file = "grpcio-1.43.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:80398e9fb598060fa41050d1220f5a2440fe74ff082c36dda41ac3215ebb5ddd"}, + {file = "grpcio-1.43.0-cp36-cp36m-win32.whl", hash = "sha256:0110310eff07bb69782f53b7a947490268c4645de559034c43c0a635612e250f"}, + {file = "grpcio-1.43.0-cp36-cp36m-win_amd64.whl", hash = "sha256:45401d00f2ee46bde75618bf33e9df960daa7980e6e0e7328047191918c98504"}, + {file = "grpcio-1.43.0-cp37-cp37m-linux_armv7l.whl", hash = "sha256:af78ac55933811e6a25141336b1f2d5e0659c2f568d44d20539b273792563ca7"}, + {file = "grpcio-1.43.0-cp37-cp37m-macosx_10_10_x86_64.whl", hash = "sha256:8b2b9dc4d7897566723b77422e11c009a0ebd397966b165b21b89a62891a9fdf"}, + {file = "grpcio-1.43.0-cp37-cp37m-manylinux2010_i686.whl", hash = "sha256:77ef653f966934b3bfdd00e4f2064b68880eb40cf09b0b99edfa5ee22a44f559"}, + {file = "grpcio-1.43.0-cp37-cp37m-manylinux2010_x86_64.whl", hash = "sha256:e95b5d62ec26d0cd0b90c202d73e7cb927c369c3358e027225239a4e354967dc"}, + {file = "grpcio-1.43.0-cp37-cp37m-manylinux_2_17_aarch64.whl", hash = "sha256:04239e8f71db832c26bbbedb4537b37550a39d77681d748ab4678e58dd6455d6"}, + {file = "grpcio-1.43.0-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:4b4a7152187a49767a47d1413edde2304c96f41f7bc92cc512e230dfd0fba095"}, + {file = "grpcio-1.43.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b8cc936a29c65ab39714e1ba67a694c41218f98b6e2a64efb83f04d9abc4386b"}, + {file = "grpcio-1.43.0-cp37-cp37m-win32.whl", hash = "sha256:577e024c8dd5f27cd98ba850bc4e890f07d4b5942e5bc059a3d88843a2f48f66"}, + {file = "grpcio-1.43.0-cp37-cp37m-win_amd64.whl", hash = "sha256:138f57e3445d4a48d9a8a5af1538fdaafaa50a0a3c243f281d8df0edf221dc02"}, + {file = "grpcio-1.43.0-cp38-cp38-linux_armv7l.whl", hash = "sha256:08cf25f2936629db062aeddbb594bd76b3383ab0ede75ef0461a3b0bc3a2c150"}, + {file = "grpcio-1.43.0-cp38-cp38-macosx_10_10_x86_64.whl", hash = "sha256:01f4b887ed703fe82ebe613e1d2dadea517891725e17e7a6134dcd00352bd28c"}, + {file = "grpcio-1.43.0-cp38-cp38-manylinux2010_i686.whl", hash = "sha256:0aa8285f284338eb68962fe1a830291db06f366ea12f213399b520c062b01f65"}, + {file = "grpcio-1.43.0-cp38-cp38-manylinux2010_x86_64.whl", hash = "sha256:0edbfeb6729aa9da33ce7e28fb7703b3754934115454ae45e8cc1db601756fd3"}, + {file = "grpcio-1.43.0-cp38-cp38-manylinux_2_17_aarch64.whl", hash = "sha256:c354017819201053d65212befd1dcb65c2d91b704d8977e696bae79c47cd2f82"}, + {file = "grpcio-1.43.0-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:50cfb7e1067ee5e00b8ab100a6b7ea322d37ec6672c0455106520b5891c4b5f5"}, + {file = "grpcio-1.43.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:57f1aeb65ed17dfb2f6cd717cc109910fe395133af7257a9c729c0b9604eac10"}, + {file = "grpcio-1.43.0-cp38-cp38-win32.whl", hash = "sha256:fa26a8bbb3fe57845acb1329ff700d5c7eaf06414c3e15f4cb8923f3a466ef64"}, + {file = "grpcio-1.43.0-cp38-cp38-win_amd64.whl", hash = "sha256:ade8b79a6b6aea68adb9d4bfeba5d647667d842202c5d8f3ba37ac1dc8e5c09c"}, + {file = "grpcio-1.43.0-cp39-cp39-linux_armv7l.whl", hash = "sha256:124e718faf96fe44c98b05f3f475076be8b5198bb4c52a13208acf88a8548ba9"}, + {file = "grpcio-1.43.0-cp39-cp39-macosx_10_10_x86_64.whl", hash = "sha256:2f96142d0abc91290a63ba203f01649e498302b1b6007c67bad17f823ecde0cf"}, + {file = "grpcio-1.43.0-cp39-cp39-manylinux2010_i686.whl", hash = "sha256:31e6e489ccd8f08884b9349a39610982df48535881ec34f05a11c6e6b6ebf9d0"}, + {file = "grpcio-1.43.0-cp39-cp39-manylinux2010_x86_64.whl", hash = "sha256:0e731f660e1e68238f56f4ce11156f02fd06dc58bc7834778d42c0081d4ef5ad"}, + {file = "grpcio-1.43.0-cp39-cp39-manylinux_2_17_aarch64.whl", hash = "sha256:1f16725a320460435a8a5339d8b06c4e00d307ab5ad56746af2e22b5f9c50932"}, + {file = "grpcio-1.43.0-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a4b4543e13acb4806917d883d0f70f21ba93b29672ea81f4aaba14821aaf9bb0"}, + {file = "grpcio-1.43.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:594aaa0469f4fca7773e80d8c27bf1298e7bbce5f6da0f084b07489a708f16ab"}, + {file = "grpcio-1.43.0-cp39-cp39-win32.whl", hash = "sha256:5449ae564349e7a738b8c38583c0aad954b0d5d1dd3cea68953bfc32eaee11e3"}, + {file = "grpcio-1.43.0-cp39-cp39-win_amd64.whl", hash = "sha256:bdf41550815a831384d21a498b20597417fd31bd084deb17d31ceb39ad9acc79"}, + {file = "grpcio-1.43.0.tar.gz", hash = "sha256:735d9a437c262ab039d02defddcb9f8f545d7009ae61c0114e19dda3843febe5"}, +] +h11 = [ + {file = "h11-0.12.0-py3-none-any.whl", hash = "sha256:36a3cb8c0a032f56e2da7084577878a035d3b61d104230d4bd49c0c6b555a9c6"}, + {file = "h11-0.12.0.tar.gz", hash = "sha256:47222cb6067e4a307d535814917cd98fd0a57b6788ce715755fa2b6c28b56042"}, +] +h5py = [ + {file = "h5py-2.10.0-cp27-cp27m-macosx_10_6_intel.whl", hash = "sha256:ecf4d0b56ee394a0984de15bceeb97cbe1fe485f1ac205121293fc44dcf3f31f"}, + {file = "h5py-2.10.0-cp27-cp27m-manylinux1_i686.whl", hash = "sha256:86868dc07b9cc8cb7627372a2e6636cdc7a53b7e2854ad020c9e9d8a4d3fd0f5"}, + {file = "h5py-2.10.0-cp27-cp27m-manylinux1_x86_64.whl", hash = "sha256:aac4b57097ac29089f179bbc2a6e14102dd210618e94d77ee4831c65f82f17c0"}, + {file = "h5py-2.10.0-cp27-cp27m-win32.whl", hash = "sha256:7be5754a159236e95bd196419485343e2b5875e806fe68919e087b6351f40a70"}, + {file = "h5py-2.10.0-cp27-cp27m-win_amd64.whl", hash = "sha256:13c87efa24768a5e24e360a40e0bc4c49bcb7ce1bb13a3a7f9902cec302ccd36"}, + {file = "h5py-2.10.0-cp27-cp27mu-manylinux1_i686.whl", hash = "sha256:79b23f47c6524d61f899254f5cd5e486e19868f1823298bc0c29d345c2447172"}, + {file = "h5py-2.10.0-cp27-cp27mu-manylinux1_x86_64.whl", hash = "sha256:cbf28ae4b5af0f05aa6e7551cee304f1d317dbed1eb7ac1d827cee2f1ef97a99"}, + {file = "h5py-2.10.0-cp34-cp34m-manylinux1_i686.whl", hash = "sha256:c0d4b04bbf96c47b6d360cd06939e72def512b20a18a8547fa4af810258355d5"}, + {file = "h5py-2.10.0-cp34-cp34m-manylinux1_x86_64.whl", hash = "sha256:549ad124df27c056b2e255ea1c44d30fb7a17d17676d03096ad5cd85edb32dc1"}, + {file = "h5py-2.10.0-cp35-cp35m-macosx_10_6_intel.whl", hash = "sha256:a5f82cd4938ff8761d9760af3274acf55afc3c91c649c50ab18fcff5510a14a5"}, + {file = "h5py-2.10.0-cp35-cp35m-manylinux1_i686.whl", hash = "sha256:3dad1730b6470fad853ef56d755d06bb916ee68a3d8272b3bab0c1ddf83bb99e"}, + {file = "h5py-2.10.0-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:063947eaed5f271679ed4ffa36bb96f57bc14f44dd4336a827d9a02702e6ce6b"}, + {file = "h5py-2.10.0-cp35-cp35m-win32.whl", hash = "sha256:c54a2c0dd4957776ace7f95879d81582298c5daf89e77fb8bee7378f132951de"}, + {file = "h5py-2.10.0-cp35-cp35m-win_amd64.whl", hash = "sha256:6998be619c695910cb0effe5eb15d3a511d3d1a5d217d4bd0bebad1151ec2262"}, + {file = "h5py-2.10.0-cp36-cp36m-macosx_10_6_intel.whl", hash = "sha256:ff7d241f866b718e4584fa95f520cb19405220c501bd3a53ee11871ba5166ea2"}, + {file = "h5py-2.10.0-cp36-cp36m-manylinux1_i686.whl", hash = "sha256:54817b696e87eb9e403e42643305f142cd8b940fe9b3b490bbf98c3b8a894cf4"}, + {file = "h5py-2.10.0-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:d3c59549f90a891691991c17f8e58c8544060fdf3ccdea267100fa5f561ff62f"}, + {file = "h5py-2.10.0-cp36-cp36m-win32.whl", hash = "sha256:d7ae7a0576b06cb8e8a1c265a8bc4b73d05fdee6429bffc9a26a6eb531e79d72"}, + {file = "h5py-2.10.0-cp36-cp36m-win_amd64.whl", hash = "sha256:bffbc48331b4a801d2f4b7dac8a72609f0b10e6e516e5c480a3e3241e091c878"}, + {file = "h5py-2.10.0-cp37-cp37m-macosx_10_6_intel.whl", hash = "sha256:51ae56894c6c93159086ffa2c94b5b3388c0400548ab26555c143e7cfa05b8e5"}, + {file = "h5py-2.10.0-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:16ead3c57141101e3296ebeed79c9c143c32bdd0e82a61a2fc67e8e6d493e9d1"}, + {file = "h5py-2.10.0-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:f0e25bb91e7a02efccb50aba6591d3fe2c725479e34769802fcdd4076abfa917"}, + {file = "h5py-2.10.0-cp37-cp37m-win32.whl", hash = "sha256:f23951a53d18398ef1344c186fb04b26163ca6ce449ebd23404b153fd111ded9"}, + {file = "h5py-2.10.0-cp37-cp37m-win_amd64.whl", hash = "sha256:8bb1d2de101f39743f91512a9750fb6c351c032e5cd3204b4487383e34da7f75"}, + {file = "h5py-2.10.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:64f74da4a1dd0d2042e7d04cf8294e04ddad686f8eba9bb79e517ae582f6668d"}, + {file = "h5py-2.10.0-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:d35f7a3a6cefec82bfdad2785e78359a0e6a5fbb3f605dd5623ce88082ccd681"}, + {file = "h5py-2.10.0-cp38-cp38-win32.whl", hash = "sha256:6ef7ab1089e3ef53ca099038f3c0a94d03e3560e6aff0e9d6c64c55fb13fc681"}, + {file = "h5py-2.10.0-cp38-cp38-win_amd64.whl", hash = "sha256:769e141512b54dee14ec76ed354fcacfc7d97fea5a7646b709f7400cf1838630"}, + {file = "h5py-2.10.0.tar.gz", hash = "sha256:84412798925dc870ffd7107f045d7659e60f5d46d1c70c700375248bf6bf512d"}, +] +identify = [ + {file = "identify-2.4.1-py2.py3-none-any.whl", hash = "sha256:0192893ff68b03d37fed553e261d4a22f94ea974093aefb33b29df2ff35fed3c"}, + {file = "identify-2.4.1.tar.gz", hash = "sha256:64d4885e539f505dd8ffb5e93c142a1db45480452b1594cacd3e91dca9a984e9"}, +] +idna = [ + {file = "idna-3.3-py3-none-any.whl", hash = "sha256:84d9dd047ffa80596e0f246e2eab0b391788b0503584e8945f2368256d2735ff"}, + {file = "idna-3.3.tar.gz", hash = "sha256:9d643ff0a55b762d5cdb124b8eaa99c66322e2157b69160bc32796e824360e6d"}, +] +imagesize = [ + {file = "imagesize-1.3.0-py2.py3-none-any.whl", hash = "sha256:1db2f82529e53c3e929e8926a1fa9235aa82d0bd0c580359c67ec31b2fddaa8c"}, + {file = "imagesize-1.3.0.tar.gz", hash = "sha256:cd1750d452385ca327479d45b64d9c7729ecf0b3969a58148298c77092261f9d"}, +] +importlib-metadata = [ + {file = "importlib_metadata-1.7.0-py2.py3-none-any.whl", hash = "sha256:dc15b2969b4ce36305c51eebe62d418ac7791e9a157911d58bfb1f9ccd8e2070"}, + {file = "importlib_metadata-1.7.0.tar.gz", hash = "sha256:90bb658cdbbf6d1735b6341ce708fc7024a3e14e99ffdc5783edea9f9b077f83"}, +] +iniconfig = [ + {file = "iniconfig-1.1.1-py2.py3-none-any.whl", hash = "sha256:011e24c64b7f47f6ebd835bb12a743f2fbe9a26d4cecaa7f53bc4f35ee9da8b3"}, + {file = "iniconfig-1.1.1.tar.gz", hash = "sha256:bc3af051d7d14b2ee5ef9969666def0cd1a000e121eaea580d4a313df4b37f32"}, +] +ipython = [ + {file = "ipython-7.30.1-py3-none-any.whl", hash = "sha256:fc60ef843e0863dd4e24ab2bb5698f071031332801ecf8d1aeb4fb622056545c"}, + {file = "ipython-7.30.1.tar.gz", hash = "sha256:cb6aef731bf708a7727ab6cde8df87f0281b1427d41e65d62d4b68934fa54e97"}, +] +isort = [ + {file = "isort-5.10.1-py3-none-any.whl", hash = "sha256:6f62d78e2f89b4500b080fe3a81690850cd254227f27f75c3a0c491a1f351ba7"}, + {file = "isort-5.10.1.tar.gz", hash = "sha256:e8443a5e7a020e9d7f97f1d7d9cd17c88bcb3bc7e218bf9cf5095fe550be2951"}, +] +jedi = [ + {file = "jedi-0.18.1-py2.py3-none-any.whl", hash = "sha256:637c9635fcf47945ceb91cd7f320234a7be540ded6f3e99a50cb6febdfd1ba8d"}, + {file = "jedi-0.18.1.tar.gz", hash = "sha256:74137626a64a99c8eb6ae5832d99b3bdd7d29a3850fe2aa80a4126b2a7d949ab"}, +] +jeepney = [ + {file = "jeepney-0.7.1-py3-none-any.whl", hash = "sha256:1b5a0ea5c0e7b166b2f5895b91a08c14de8915afda4407fb5022a195224958ac"}, + {file = "jeepney-0.7.1.tar.gz", hash = "sha256:fa9e232dfa0c498bd0b8a3a73b8d8a31978304dcef0515adc859d4e096f96f4f"}, +] +jinja2 = [ + {file = "Jinja2-3.0.3-py3-none-any.whl", hash = "sha256:077ce6014f7b40d03b47d1f1ca4b0fc8328a692bd284016f806ed0eaca390ad8"}, + {file = "Jinja2-3.0.3.tar.gz", hash = "sha256:611bb273cd68f3b993fabdc4064fc858c5b47a973cb5aa7999ec1ba405c87cd7"}, +] +joblib = [ + {file = "joblib-0.16.0-py3-none-any.whl", hash = "sha256:d348c5d4ae31496b2aa060d6d9b787864dd204f9480baaa52d18850cb43e9f49"}, + {file = "joblib-0.16.0.tar.gz", hash = "sha256:8f52bf24c64b608bf0b2563e0e47d6fcf516abc8cfafe10cfd98ad66d94f92d6"}, +] +keras-preprocessing = [ + {file = "Keras_Preprocessing-1.1.2-py2.py3-none-any.whl", hash = "sha256:7b82029b130ff61cc99b55f3bd27427df4838576838c5b2f65940e4fcec99a7b"}, + {file = "Keras_Preprocessing-1.1.2.tar.gz", hash = "sha256:add82567c50c8bc648c14195bf544a5ce7c1f76761536956c3d2978970179ef3"}, +] +keras-tuner = [ + {file = "keras-tuner-1.0.4.tar.gz", hash = "sha256:5864db0320ce61b2e11e4533a9775a9bf213dbee97720409327dc41c71d0b88b"}, + {file = "keras_tuner-1.0.4-py3-none-any.whl", hash = "sha256:360502c2944aa03d0ed984d5d806aa515ef054d4decf38bb834ed17a5db65d12"}, +] +keyring = [ + {file = "keyring-22.3.0-py3-none-any.whl", hash = "sha256:2bc8363ebdd63886126a012057a85c8cb6e143877afa02619ac7dbc9f38a207b"}, + {file = "keyring-22.3.0.tar.gz", hash = "sha256:16927a444b2c73f983520a48dec79ddab49fe76429ea05b8d528d778c8339522"}, +] +kt-legacy = [ + {file = "kt-legacy-1.0.4.tar.gz", hash = "sha256:a94112e42a50e7cc3aad31f3287aa384c23555ea1432c55b5823852e09e706cf"}, + {file = "kt_legacy-1.0.4-py3-none-any.whl", hash = "sha256:8b6eaff78b01b3cf1d71390cbcc6498208433e2ae0ce4d3c6e072f980b9fc625"}, +] +lazy-object-proxy = [ + {file = "lazy-object-proxy-1.7.1.tar.gz", hash = "sha256:d609c75b986def706743cdebe5e47553f4a5a1da9c5ff66d76013ef396b5a8a4"}, + {file = "lazy_object_proxy-1.7.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:bb8c5fd1684d60a9902c60ebe276da1f2281a318ca16c1d0a96db28f62e9166b"}, + {file = "lazy_object_proxy-1.7.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a57d51ed2997e97f3b8e3500c984db50a554bb5db56c50b5dab1b41339b37e36"}, + {file = "lazy_object_proxy-1.7.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fd45683c3caddf83abbb1249b653a266e7069a09f486daa8863fb0e7496a9fdb"}, + {file = "lazy_object_proxy-1.7.1-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:8561da8b3dd22d696244d6d0d5330618c993a215070f473b699e00cf1f3f6443"}, + {file = "lazy_object_proxy-1.7.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:fccdf7c2c5821a8cbd0a9440a456f5050492f2270bd54e94360cac663398739b"}, + {file = "lazy_object_proxy-1.7.1-cp310-cp310-win32.whl", hash = "sha256:898322f8d078f2654d275124a8dd19b079080ae977033b713f677afcfc88e2b9"}, + {file = "lazy_object_proxy-1.7.1-cp310-cp310-win_amd64.whl", hash = "sha256:85b232e791f2229a4f55840ed54706110c80c0a210d076eee093f2b2e33e1bfd"}, + {file = "lazy_object_proxy-1.7.1-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:46ff647e76f106bb444b4533bb4153c7370cdf52efc62ccfc1a28bdb3cc95442"}, + {file = "lazy_object_proxy-1.7.1-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:12f3bb77efe1367b2515f8cb4790a11cffae889148ad33adad07b9b55e0ab22c"}, + {file = "lazy_object_proxy-1.7.1-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c19814163728941bb871240d45c4c30d33b8a2e85972c44d4e63dd7107faba44"}, + {file = "lazy_object_proxy-1.7.1-cp36-cp36m-musllinux_1_1_aarch64.whl", hash = "sha256:e40f2013d96d30217a51eeb1db28c9ac41e9d0ee915ef9d00da639c5b63f01a1"}, + {file = "lazy_object_proxy-1.7.1-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:2052837718516a94940867e16b1bb10edb069ab475c3ad84fd1e1a6dd2c0fcfc"}, + {file = "lazy_object_proxy-1.7.1-cp36-cp36m-win32.whl", hash = "sha256:6a24357267aa976abab660b1d47a34aaf07259a0c3859a34e536f1ee6e76b5bb"}, + {file = "lazy_object_proxy-1.7.1-cp36-cp36m-win_amd64.whl", hash = "sha256:6aff3fe5de0831867092e017cf67e2750c6a1c7d88d84d2481bd84a2e019ec35"}, + {file = "lazy_object_proxy-1.7.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:6a6e94c7b02641d1311228a102607ecd576f70734dc3d5e22610111aeacba8a0"}, + {file = "lazy_object_proxy-1.7.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c4ce15276a1a14549d7e81c243b887293904ad2d94ad767f42df91e75fd7b5b6"}, + {file = "lazy_object_proxy-1.7.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e368b7f7eac182a59ff1f81d5f3802161932a41dc1b1cc45c1f757dc876b5d2c"}, + {file = "lazy_object_proxy-1.7.1-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:6ecbb350991d6434e1388bee761ece3260e5228952b1f0c46ffc800eb313ff42"}, + {file = "lazy_object_proxy-1.7.1-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:553b0f0d8dbf21890dd66edd771f9b1b5f51bd912fa5f26de4449bfc5af5e029"}, + {file = "lazy_object_proxy-1.7.1-cp37-cp37m-win32.whl", hash = "sha256:c7a683c37a8a24f6428c28c561c80d5f4fd316ddcf0c7cab999b15ab3f5c5c69"}, + {file = "lazy_object_proxy-1.7.1-cp37-cp37m-win_amd64.whl", hash = "sha256:df2631f9d67259dc9620d831384ed7732a198eb434eadf69aea95ad18c587a28"}, + {file = "lazy_object_proxy-1.7.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:07fa44286cda977bd4803b656ffc1c9b7e3bc7dff7d34263446aec8f8c96f88a"}, + {file = "lazy_object_proxy-1.7.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4dca6244e4121c74cc20542c2ca39e5c4a5027c81d112bfb893cf0790f96f57e"}, + {file = "lazy_object_proxy-1.7.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:91ba172fc5b03978764d1df5144b4ba4ab13290d7bab7a50f12d8117f8630c38"}, + {file = "lazy_object_proxy-1.7.1-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:043651b6cb706eee4f91854da4a089816a6606c1428fd391573ef8cb642ae4f7"}, + {file = "lazy_object_proxy-1.7.1-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:b9e89b87c707dd769c4ea91f7a31538888aad05c116a59820f28d59b3ebfe25a"}, + {file = "lazy_object_proxy-1.7.1-cp38-cp38-win32.whl", hash = "sha256:9d166602b525bf54ac994cf833c385bfcc341b364e3ee71e3bf5a1336e677b55"}, + {file = "lazy_object_proxy-1.7.1-cp38-cp38-win_amd64.whl", hash = "sha256:8f3953eb575b45480db6568306893f0bd9d8dfeeebd46812aa09ca9579595148"}, + {file = "lazy_object_proxy-1.7.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:dd7ed7429dbb6c494aa9bc4e09d94b778a3579be699f9d67da7e6804c422d3de"}, + {file = "lazy_object_proxy-1.7.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:70ed0c2b380eb6248abdef3cd425fc52f0abd92d2b07ce26359fcbc399f636ad"}, + {file = "lazy_object_proxy-1.7.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7096a5e0c1115ec82641afbdd70451a144558ea5cf564a896294e346eb611be1"}, + {file = "lazy_object_proxy-1.7.1-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:f769457a639403073968d118bc70110e7dce294688009f5c24ab78800ae56dc8"}, + {file = "lazy_object_proxy-1.7.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:39b0e26725c5023757fc1ab2a89ef9d7ab23b84f9251e28f9cc114d5b59c1b09"}, + {file = "lazy_object_proxy-1.7.1-cp39-cp39-win32.whl", hash = "sha256:2130db8ed69a48a3440103d4a520b89d8a9405f1b06e2cc81640509e8bf6548f"}, + {file = "lazy_object_proxy-1.7.1-cp39-cp39-win_amd64.whl", hash = "sha256:677ea950bef409b47e51e733283544ac3d660b709cfce7b187f5ace137960d61"}, + {file = "lazy_object_proxy-1.7.1-pp37.pp38-none-any.whl", hash = "sha256:d66906d5785da8e0be7360912e99c9188b70f52c422f9fc18223347235691a84"}, +] +markdown = [ + {file = "Markdown-3.3.5-py3-none-any.whl", hash = "sha256:0d2d09f75cb8d1ffc6770c65c61770b23a61708101f47bda416a002a0edbc480"}, + {file = "Markdown-3.3.5.tar.gz", hash = "sha256:26e9546bfbcde5fcd072bd8f612c9c1b6e2677cb8aadbdf65206674f46dde069"}, +] +markupsafe = [ + {file = "MarkupSafe-2.0.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:d8446c54dc28c01e5a2dbac5a25f071f6653e6e40f3a8818e8b45d790fe6ef53"}, + {file = "MarkupSafe-2.0.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:36bc903cbb393720fad60fc28c10de6acf10dc6cc883f3e24ee4012371399a38"}, + {file = "MarkupSafe-2.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2d7d807855b419fc2ed3e631034685db6079889a1f01d5d9dac950f764da3dad"}, + {file = "MarkupSafe-2.0.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:add36cb2dbb8b736611303cd3bfcee00afd96471b09cda130da3581cbdc56a6d"}, + {file = "MarkupSafe-2.0.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:168cd0a3642de83558a5153c8bd34f175a9a6e7f6dc6384b9655d2697312a646"}, + {file = "MarkupSafe-2.0.1-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:4dc8f9fb58f7364b63fd9f85013b780ef83c11857ae79f2feda41e270468dd9b"}, + {file = "MarkupSafe-2.0.1-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:20dca64a3ef2d6e4d5d615a3fd418ad3bde77a47ec8a23d984a12b5b4c74491a"}, + {file = "MarkupSafe-2.0.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:cdfba22ea2f0029c9261a4bd07e830a8da012291fbe44dc794e488b6c9bb353a"}, + {file = "MarkupSafe-2.0.1-cp310-cp310-win32.whl", hash = "sha256:99df47edb6bda1249d3e80fdabb1dab8c08ef3975f69aed437cb69d0a5de1e28"}, + {file = "MarkupSafe-2.0.1-cp310-cp310-win_amd64.whl", hash = "sha256:e0f138900af21926a02425cf736db95be9f4af72ba1bb21453432a07f6082134"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:f9081981fe268bd86831e5c75f7de206ef275defcb82bc70740ae6dc507aee51"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-manylinux1_i686.whl", hash = "sha256:0955295dd5eec6cb6cc2fe1698f4c6d84af2e92de33fbcac4111913cd100a6ff"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:0446679737af14f45767963a1a9ef7620189912317d095f2d9ffa183a4d25d2b"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-manylinux2010_i686.whl", hash = "sha256:f826e31d18b516f653fe296d967d700fddad5901ae07c622bb3705955e1faa94"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-manylinux2010_x86_64.whl", hash = "sha256:fa130dd50c57d53368c9d59395cb5526eda596d3ffe36666cd81a44d56e48872"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-manylinux2014_aarch64.whl", hash = "sha256:905fec760bd2fa1388bb5b489ee8ee5f7291d692638ea5f67982d968366bef9f"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:bf5d821ffabf0ef3533c39c518f3357b171a1651c1ff6827325e4489b0e46c3c"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:0d4b31cc67ab36e3392bbf3862cfbadac3db12bdd8b02a2731f509ed5b829724"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:baa1a4e8f868845af802979fcdbf0bb11f94f1cb7ced4c4b8a351bb60d108145"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-musllinux_1_1_aarch64.whl", hash = "sha256:deb993cacb280823246a026e3b2d81c493c53de6acfd5e6bfe31ab3402bb37dd"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-musllinux_1_1_i686.whl", hash = "sha256:63f3268ba69ace99cab4e3e3b5840b03340efed0948ab8f78d2fd87ee5442a4f"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:8d206346619592c6200148b01a2142798c989edcb9c896f9ac9722a99d4e77e6"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-win32.whl", hash = "sha256:6c4ca60fa24e85fe25b912b01e62cb969d69a23a5d5867682dd3e80b5b02581d"}, + {file = "MarkupSafe-2.0.1-cp36-cp36m-win_amd64.whl", hash = "sha256:b2f4bf27480f5e5e8ce285a8c8fd176c0b03e93dcc6646477d4630e83440c6a9"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:0717a7390a68be14b8c793ba258e075c6f4ca819f15edfc2a3a027c823718567"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:6557b31b5e2c9ddf0de32a691f2312a32f77cd7681d8af66c2692efdbef84c18"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:49e3ceeabbfb9d66c3aef5af3a60cc43b85c33df25ce03d0031a608b0a8b2e3f"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-manylinux2010_i686.whl", hash = "sha256:d7f9850398e85aba693bb640262d3611788b1f29a79f0c93c565694658f4071f"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-manylinux2010_x86_64.whl", hash = "sha256:6a7fae0dd14cf60ad5ff42baa2e95727c3d81ded453457771d02b7d2b3f9c0c2"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-manylinux2014_aarch64.whl", hash = "sha256:b7f2d075102dc8c794cbde1947378051c4e5180d52d276987b8d28a3bd58c17d"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e9936f0b261d4df76ad22f8fee3ae83b60d7c3e871292cd42f40b81b70afae85"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:2a7d351cbd8cfeb19ca00de495e224dea7e7d919659c2841bbb7f420ad03e2d6"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:60bf42e36abfaf9aff1f50f52644b336d4f0a3fd6d8a60ca0d054ac9f713a864"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:d6c7ebd4e944c85e2c3421e612a7057a2f48d478d79e61800d81468a8d842207"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:f0567c4dc99f264f49fe27da5f735f414c4e7e7dd850cfd8e69f0862d7c74ea9"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:89c687013cb1cd489a0f0ac24febe8c7a666e6e221b783e53ac50ebf68e45d86"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-win32.whl", hash = "sha256:a30e67a65b53ea0a5e62fe23682cfe22712e01f453b95233b25502f7c61cb415"}, + {file = "MarkupSafe-2.0.1-cp37-cp37m-win_amd64.whl", hash = "sha256:611d1ad9a4288cf3e3c16014564df047fe08410e628f89805e475368bd304914"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:5bb28c636d87e840583ee3adeb78172efc47c8b26127267f54a9c0ec251d41a9"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:be98f628055368795d818ebf93da628541e10b75b41c559fdf36d104c5787066"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-manylinux1_i686.whl", hash = "sha256:1d609f577dc6e1aa17d746f8bd3c31aa4d258f4070d61b2aa5c4166c1539de35"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:7d91275b0245b1da4d4cfa07e0faedd5b0812efc15b702576d103293e252af1b"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-manylinux2010_i686.whl", hash = "sha256:01a9b8ea66f1658938f65b93a85ebe8bc016e6769611be228d797c9d998dd298"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-manylinux2010_x86_64.whl", hash = "sha256:47ab1e7b91c098ab893b828deafa1203de86d0bc6ab587b160f78fe6c4011f75"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:97383d78eb34da7e1fa37dd273c20ad4320929af65d156e35a5e2d89566d9dfb"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6fcf051089389abe060c9cd7caa212c707e58153afa2c649f00346ce6d260f1b"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:5855f8438a7d1d458206a2466bf82b0f104a3724bf96a1c781ab731e4201731a"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:3dd007d54ee88b46be476e293f48c85048603f5f516008bee124ddd891398ed6"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:aca6377c0cb8a8253e493c6b451565ac77e98c2951c45f913e0b52facdcff83f"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:04635854b943835a6ea959e948d19dcd311762c5c0c6e1f0e16ee57022669194"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:6300b8454aa6930a24b9618fbb54b5a68135092bc666f7b06901f897fa5c2fee"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-win32.whl", hash = "sha256:023cb26ec21ece8dc3907c0e8320058b2e0cb3c55cf9564da612bc325bed5e64"}, + {file = "MarkupSafe-2.0.1-cp38-cp38-win_amd64.whl", hash = "sha256:984d76483eb32f1bcb536dc27e4ad56bba4baa70be32fa87152832cdd9db0833"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:2ef54abee730b502252bcdf31b10dacb0a416229b72c18b19e24a4509f273d26"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:3c112550557578c26af18a1ccc9e090bfe03832ae994343cfdacd287db6a6ae7"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-manylinux1_i686.whl", hash = "sha256:53edb4da6925ad13c07b6d26c2a852bd81e364f95301c66e930ab2aef5b5ddd8"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-manylinux1_x86_64.whl", hash = "sha256:f5653a225f31e113b152e56f154ccbe59eeb1c7487b39b9d9f9cdb58e6c79dc5"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-manylinux2010_i686.whl", hash = "sha256:4efca8f86c54b22348a5467704e3fec767b2db12fc39c6d963168ab1d3fc9135"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-manylinux2010_x86_64.whl", hash = "sha256:ab3ef638ace319fa26553db0624c4699e31a28bb2a835c5faca8f8acf6a5a902"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:f8ba0e8349a38d3001fae7eadded3f6606f0da5d748ee53cc1dab1d6527b9509"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c47adbc92fc1bb2b3274c4b3a43ae0e4573d9fbff4f54cd484555edbf030baf1"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:37205cac2a79194e3750b0af2a5720d95f786a55ce7df90c3af697bfa100eaac"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:1f2ade76b9903f39aa442b4aadd2177decb66525062db244b35d71d0ee8599b6"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:4296f2b1ce8c86a6aea78613c34bb1a672ea0e3de9c6ba08a960efe0b0a09047"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:9f02365d4e99430a12647f09b6cc8bab61a6564363f313126f775eb4f6ef798e"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:5b6d930f030f8ed98e3e6c98ffa0652bdb82601e7a016ec2ab5d7ff23baa78d1"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-win32.whl", hash = "sha256:10f82115e21dc0dfec9ab5c0223652f7197feb168c940f3ef61563fc2d6beb74"}, + {file = "MarkupSafe-2.0.1-cp39-cp39-win_amd64.whl", hash = "sha256:693ce3f9e70a6cf7d2fb9e6c9d8b204b6b39897a2c4a1aa65728d5ac97dcc1d8"}, + {file = "MarkupSafe-2.0.1.tar.gz", hash = "sha256:594c67807fb16238b30c44bdf74f36c02cdf22d1c8cda91ef8a0ed8dabf5620a"}, +] +matplotlib-inline = [ + {file = "matplotlib-inline-0.1.3.tar.gz", hash = "sha256:a04bfba22e0d1395479f866853ec1ee28eea1485c1d69a6faf00dc3e24ff34ee"}, + {file = "matplotlib_inline-0.1.3-py3-none-any.whl", hash = "sha256:aed605ba3b72462d64d475a21a9296f400a19c4f74a31b59103d2a99ffd5aa5c"}, +] +mccabe = [ + {file = "mccabe-0.6.1-py2.py3-none-any.whl", hash = "sha256:ab8a6258860da4b6677da4bd2fe5dc2c659cff31b3ee4f7f5d64e79735b80d42"}, + {file = "mccabe-0.6.1.tar.gz", hash = "sha256:dd8d182285a0fe56bace7f45b5e7d1a6ebcbf524e8f3bd87eb0f125271b8831f"}, +] +more-itertools = [ + {file = "more-itertools-8.12.0.tar.gz", hash = "sha256:7dc6ad46f05f545f900dd59e8dfb4e84a4827b97b3cfecb175ea0c7d247f6064"}, + {file = "more_itertools-8.12.0-py3-none-any.whl", hash = "sha256:43e6dd9942dffd72661a2c4ef383ad7da1e6a3e968a927ad7a6083ab410a688b"}, +] +mypy = [ + {file = "mypy-0.902-cp35-cp35m-macosx_10_9_x86_64.whl", hash = "sha256:3f12705eabdd274b98f676e3e5a89f247ea86dc1af48a2d5a2b080abac4e1243"}, + {file = "mypy-0.902-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:2f9fedc1f186697fda191e634ac1d02f03d4c260212ccb018fabbb6d4b03eee8"}, + {file = "mypy-0.902-cp35-cp35m-manylinux2010_x86_64.whl", hash = "sha256:0756529da2dd4d53d26096b7969ce0a47997123261a5432b48cc6848a2cb0bd4"}, + {file = "mypy-0.902-cp35-cp35m-win_amd64.whl", hash = "sha256:68a098c104ae2b75e946b107ef69dd8398d54cb52ad57580dfb9fc78f7f997f0"}, + {file = "mypy-0.902-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:cd01c599cf9f897b6b6c6b5d8b182557fb7d99326bcdf5d449a0fbbb4ccee4b9"}, + {file = "mypy-0.902-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:e89880168c67cf4fde4506b80ee42f1537ad66ad366c101d388b3fd7d7ce2afd"}, + {file = "mypy-0.902-cp36-cp36m-manylinux2010_x86_64.whl", hash = "sha256:ebe2bc9cb638475f5d39068d2dbe8ae1d605bb8d8d3ff281c695df1670ab3987"}, + {file = "mypy-0.902-cp36-cp36m-win_amd64.whl", hash = "sha256:f89bfda7f0f66b789792ab64ce0978e4a991a0e4dd6197349d0767b0f1095b21"}, + {file = "mypy-0.902-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:746e0b0101b8efec34902810047f26a8c80e1efbb4fc554956d848c05ef85d76"}, + {file = "mypy-0.902-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:0190fb77e93ce971954c9e54ea61de2802065174e5e990c9d4c1d0f54fbeeca2"}, + {file = "mypy-0.902-cp37-cp37m-manylinux2010_x86_64.whl", hash = "sha256:b5dfcd22c6bab08dfeded8d5b44bdcb68c6f1ab261861e35c470b89074f78a70"}, + {file = "mypy-0.902-cp37-cp37m-win_amd64.whl", hash = "sha256:b5ba1f0d5f9087e03bf5958c28d421a03a4c1ad260bf81556195dffeccd979c4"}, + {file = "mypy-0.902-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:9ef5355eaaf7a23ab157c21a44c614365238a7bdb3552ec3b80c393697d974e1"}, + {file = "mypy-0.902-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:517e7528d1be7e187a5db7f0a3e479747307c1b897d9706b1c662014faba3116"}, + {file = "mypy-0.902-cp38-cp38-manylinux2010_x86_64.whl", hash = "sha256:fd634bc17b1e2d6ce716f0e43446d0d61cdadb1efcad5c56ca211c22b246ebc8"}, + {file = "mypy-0.902-cp38-cp38-win_amd64.whl", hash = "sha256:fc4d63da57ef0e8cd4ab45131f3fe5c286ce7dd7f032650d0fbc239c6190e167"}, + {file = "mypy-0.902-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:353aac2ce41ddeaf7599f1c73fed2b75750bef3b44b6ad12985a991bc002a0da"}, + {file = "mypy-0.902-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:ae94c31bb556ddb2310e4f913b706696ccbd43c62d3331cd3511caef466871d2"}, + {file = "mypy-0.902-cp39-cp39-manylinux1_x86_64.whl", hash = "sha256:8be7bbd091886bde9fcafed8dd089a766fa76eb223135fe5c9e9798f78023a20"}, + {file = "mypy-0.902-cp39-cp39-manylinux2010_x86_64.whl", hash = "sha256:4efc67b9b3e2fddbe395700f91d5b8deb5980bfaaccb77b306310bd0b9e002eb"}, + {file = "mypy-0.902-cp39-cp39-win_amd64.whl", hash = "sha256:9f1d74eeb3f58c7bd3f3f92b8f63cb1678466a55e2c4612bf36909105d0724ab"}, + {file = "mypy-0.902-py3-none-any.whl", hash = "sha256:a26d0e53e90815c765f91966442775cf03b8a7514a4e960de7b5320208b07269"}, + {file = "mypy-0.902.tar.gz", hash = "sha256:9236c21194fde5df1b4d8ebc2ef2c1f2a5dc7f18bcbea54274937cae2e20a01c"}, +] +mypy-extensions = [ + {file = "mypy_extensions-0.4.3-py2.py3-none-any.whl", hash = "sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d"}, + {file = "mypy_extensions-0.4.3.tar.gz", hash = "sha256:2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8"}, +] +nodeenv = [ + {file = "nodeenv-1.6.0-py2.py3-none-any.whl", hash = "sha256:621e6b7076565ddcacd2db0294c0381e01fd28945ab36bcf00f41c5daf63bef7"}, + {file = "nodeenv-1.6.0.tar.gz", hash = "sha256:3ef13ff90291ba2a4a7a4ff9a979b63ffdd00a464dbe04acf0ea6471517a4c2b"}, +] +numpy = [ + {file = "numpy-1.18.5-cp35-cp35m-macosx_10_9_intel.whl", hash = "sha256:e91d31b34fc7c2c8f756b4e902f901f856ae53a93399368d9a0dc7be17ed2ca0"}, + {file = "numpy-1.18.5-cp35-cp35m-manylinux1_i686.whl", hash = "sha256:7d42ab8cedd175b5ebcb39b5208b25ba104842489ed59fbb29356f671ac93583"}, + {file = "numpy-1.18.5-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:a78e438db8ec26d5d9d0e584b27ef25c7afa5a182d1bf4d05e313d2d6d515271"}, + {file = "numpy-1.18.5-cp35-cp35m-win32.whl", hash = "sha256:a87f59508c2b7ceb8631c20630118cc546f1f815e034193dc72390db038a5cb3"}, + {file = "numpy-1.18.5-cp35-cp35m-win_amd64.whl", hash = "sha256:965df25449305092b23d5145b9bdaeb0149b6e41a77a7d728b1644b3c99277c1"}, + {file = "numpy-1.18.5-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:ac792b385d81151bae2a5a8adb2b88261ceb4976dbfaaad9ce3a200e036753dc"}, + {file = "numpy-1.18.5-cp36-cp36m-manylinux1_i686.whl", hash = "sha256:ef627986941b5edd1ed74ba89ca43196ed197f1a206a3f18cc9faf2fb84fd675"}, + {file = "numpy-1.18.5-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:f718a7949d1c4f622ff548c572e0c03440b49b9531ff00e4ed5738b459f011e8"}, + {file = "numpy-1.18.5-cp36-cp36m-win32.whl", hash = "sha256:4064f53d4cce69e9ac613256dc2162e56f20a4e2d2086b1956dd2fcf77b7fac5"}, + {file = "numpy-1.18.5-cp36-cp36m-win_amd64.whl", hash = "sha256:b03b2c0badeb606d1232e5f78852c102c0a7989d3a534b3129e7856a52f3d161"}, + {file = "numpy-1.18.5-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:a7acefddf994af1aeba05bbbafe4ba983a187079f125146dc5859e6d817df824"}, + {file = "numpy-1.18.5-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:cd49930af1d1e49a812d987c2620ee63965b619257bd76eaaa95870ca08837cf"}, + {file = "numpy-1.18.5-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:b39321f1a74d1f9183bf1638a745b4fd6fe80efbb1f6b32b932a588b4bc7695f"}, + {file = "numpy-1.18.5-cp37-cp37m-win32.whl", hash = "sha256:cae14a01a159b1ed91a324722d746523ec757357260c6804d11d6147a9e53e3f"}, + {file = "numpy-1.18.5-cp37-cp37m-win_amd64.whl", hash = "sha256:0172304e7d8d40e9e49553901903dc5f5a49a703363ed756796f5808a06fc233"}, + {file = "numpy-1.18.5-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:e15b382603c58f24265c9c931c9a45eebf44fe2e6b4eaedbb0d025ab3255228b"}, + {file = "numpy-1.18.5-cp38-cp38-manylinux1_i686.whl", hash = "sha256:3676abe3d621fc467c4c1469ee11e395c82b2d6b5463a9454e37fe9da07cd0d7"}, + {file = "numpy-1.18.5-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:4674f7d27a6c1c52a4d1aa5f0881f1eff840d2206989bae6acb1c7668c02ebfb"}, + {file = "numpy-1.18.5-cp38-cp38-win32.whl", hash = "sha256:9c9d6531bc1886454f44aa8f809268bc481295cf9740827254f53c30104f074a"}, + {file = "numpy-1.18.5-cp38-cp38-win_amd64.whl", hash = "sha256:3dd6823d3e04b5f223e3e265b4a1eae15f104f4366edd409e5a5e413a98f911f"}, + {file = "numpy-1.18.5.zip", hash = "sha256:34e96e9dae65c4839bd80012023aadd6ee2ccb73ce7fdf3074c62f301e63120b"}, +] +oauthlib = [ + {file = "oauthlib-3.1.1-py2.py3-none-any.whl", hash = "sha256:42bf6354c2ed8c6acb54d971fce6f88193d97297e18602a3a886603f9d7730cc"}, + {file = "oauthlib-3.1.1.tar.gz", hash = "sha256:8f0215fcc533dd8dd1bee6f4c412d4f0cd7297307d43ac61666389e3bc3198a3"}, +] +onnx = [ + {file = "onnx-1.10.2-cp36-cp36m-macosx_10_12_x86_64.whl", hash = "sha256:898915bcba9c1d54abef00f4ea7d60e59fdb2d21d49e7493acac40c121eca4df"}, + {file = "onnx-1.10.2-cp36-cp36m-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:86baab35fc1a317369f2a0cd3816c0eeb9036c29f9a27ed5e8f6935e67cbf0a8"}, + {file = "onnx-1.10.2-cp36-cp36m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:186abf5e9189b4b011da290c6d83d5499adefac8f6a07f5d596a192b4c911098"}, + {file = "onnx-1.10.2-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:48a747b247bc626e049341b8e8c4aeac20aa2306d6b8dff9c9e53a6b14931f1e"}, + {file = "onnx-1.10.2-cp36-cp36m-win32.whl", hash = "sha256:63aee84aed68c8e14583af48c79d99405844034043dee1efbd1937a78dfa7f6b"}, + {file = "onnx-1.10.2-cp36-cp36m-win_amd64.whl", hash = "sha256:7e59a6da6e437488059080babc9d96cde7c929cc758ffe4b0171aceaea559ada"}, + {file = "onnx-1.10.2-cp37-cp37m-macosx_10_12_x86_64.whl", hash = "sha256:358fc6f71841e30ca793a0c1bcd3d0b9c62e436e215773e77a301acb6106cbda"}, + {file = "onnx-1.10.2-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:1e2f92a77d84ae84d25ac84ec84a77b53e427cc7b2eb72ed7d56f2204f885715"}, + {file = "onnx-1.10.2-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:6205849c935837a934a9ec1fd994f1e858ad7d253e02d0bacbe4add211e4255d"}, + {file = "onnx-1.10.2-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cc830b15fe11846911fdf068460fd5f20b0f711c8b4c575c68478a6bf2884304"}, + {file = "onnx-1.10.2-cp37-cp37m-win32.whl", hash = "sha256:796fa0b80f108f2824cccf5c7298895a925aaea7831330a0bd720ceffc7be3c6"}, + {file = "onnx-1.10.2-cp37-cp37m-win_amd64.whl", hash = "sha256:24e654cca4c7285ea339fae15998dd33a5b9e57831d8ecb0bdb1f439c61c5736"}, + {file = "onnx-1.10.2-cp38-cp38-macosx_10_12_x86_64.whl", hash = "sha256:3b73128c269ef84694099dad2b06568f2672ce95761a51e0225401695dc2c136"}, + {file = "onnx-1.10.2-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:4a53055b8f13747b607dbf835914c2bd60fa7214ee719893b003ceb5fc903220"}, + {file = "onnx-1.10.2-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:a86e3f956e2a1d39772ae36d28c5b7f20fb6a883ae35971ada261b25548a8b32"}, + {file = "onnx-1.10.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:bd31e61ba95c62548543d8de2007fcb18fd2f017a9a36f712bbc08ddad1f25f4"}, + {file = "onnx-1.10.2-cp38-cp38-win32.whl", hash = "sha256:57f93db536766b1dcfeee583c02bd86c9f1c9a652253bd4f9bf189a39446de1c"}, + {file = "onnx-1.10.2-cp38-cp38-win_amd64.whl", hash = "sha256:d0a3951276ac83fde93632303ad0b3b69e10894b69b7fe5eab0361e4f4212627"}, + {file = "onnx-1.10.2-cp39-cp39-macosx_10_12_x86_64.whl", hash = "sha256:4138093cbf11e4300b7a7679aedfe1972f81abeb284a731e90dffdf3ef6c5ca3"}, + {file = "onnx-1.10.2-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:38e7d106fa98921faf909c2908bfd022eb2c594ecfbd275b60f80e0161cb8476"}, + {file = "onnx-1.10.2-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:526de93b57dd65b136bec85d5b4c6fa4455d6d817bb319b54797d29111b9c407"}, + {file = "onnx-1.10.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ce14dbe32a250b7691751e809c232b9a206da138ac055e24b9e60a1500b4d5b8"}, + {file = "onnx-1.10.2-cp39-cp39-win32.whl", hash = "sha256:253fd36cbcfcbbbe00e55dde7a09995b22fc2cc825f6de28e5ef9c47f581f264"}, + {file = "onnx-1.10.2-cp39-cp39-win_amd64.whl", hash = "sha256:0c176ef6e0c3b6bdfb69a43a66dcb8e6ba687437e302c79b4efb75027e1007dc"}, + {file = "onnx-1.10.2.tar.gz", hash = "sha256:24d73ca7dfd7e6c7339944f89554b4010719899337924fca1447d8f1b5db50d6"}, +] +onnxconverter-common = [ + {file = "onnxconverter-common-1.9.0.tar.gz", hash = "sha256:8e129c3602d1ef7619c5f3d34a53005b997137ea769362f1f8dd1ddab57ed216"}, + {file = "onnxconverter_common-1.9.0-py2.py3-none-any.whl", hash = "sha256:02b58ca3351fba4eddf8503e1421cfecd4ddcf2074aea4d58e3b2410e6f67ce5"}, +] +opt-einsum = [ + {file = "opt_einsum-3.3.0-py3-none-any.whl", hash = "sha256:2455e59e3947d3c275477df7f5205b30635e266fe6dc300e3d9f9646bfcea147"}, + {file = "opt_einsum-3.3.0.tar.gz", hash = "sha256:59f6475f77bbc37dcf7cd748519c0ec60722e91e63ca114e68821c0c54a46549"}, +] +packaging = [ + {file = "packaging-21.3-py3-none-any.whl", hash = "sha256:ef103e05f519cdc783ae24ea4e2e0f508a9c99b2d4969652eed6a2e1ea5bd522"}, + {file = "packaging-21.3.tar.gz", hash = "sha256:dd47c42927d89ab911e606518907cc2d3a1f38bbd026385970643f9c5b8ecfeb"}, +] +pandas = [ + {file = "pandas-1.1.1-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:8c9ec12c480c4d915e23ee9c8a2d8eba8509986f35f307771045c1294a2e5b73"}, + {file = "pandas-1.1.1-cp36-cp36m-manylinux1_i686.whl", hash = "sha256:e4b6c98f45695799990da328e6fd7d6187be32752ed64c2f22326ad66762d179"}, + {file = "pandas-1.1.1-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:16ae070c47474008769fc443ac765ffd88c3506b4a82966e7a605592978896f9"}, + {file = "pandas-1.1.1-cp36-cp36m-win32.whl", hash = "sha256:88930c74f69e97b17703600233c0eaf1f4f4dd10c14633d522724c5c1b963ec4"}, + {file = "pandas-1.1.1-cp36-cp36m-win_amd64.whl", hash = "sha256:fe6f1623376b616e03d51f0dd95afd862cf9a33c18cf55ce0ed4bbe1c4444391"}, + {file = "pandas-1.1.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:a81c4bf9c59010aa3efddbb6b9fc84a9b76dc0b4da2c2c2d50f06a9ef6ac0004"}, + {file = "pandas-1.1.1-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:1acc2bd7fc95e5408a4456897c2c2a1ae7c6acefe108d90479ab6d98d34fcc3d"}, + {file = "pandas-1.1.1-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:84c101d0f7bbf0d9f1be9a2f29f6fcc12415442558d067164e50a56edfb732b4"}, + {file = "pandas-1.1.1-cp37-cp37m-win32.whl", hash = "sha256:391db82ebeb886143b96b9c6c6166686c9a272d00020e4e39ad63b792542d9e2"}, + {file = "pandas-1.1.1-cp37-cp37m-win_amd64.whl", hash = "sha256:0366150fe8ee37ef89a45d3093e05026b5f895e42bbce3902ce3b6427f1b8471"}, + {file = "pandas-1.1.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:d9644ac996149b2a51325d48d77e25c911e01aa6d39dc1b64be679cd71f683ec"}, + {file = "pandas-1.1.1-cp38-cp38-manylinux1_i686.whl", hash = "sha256:41675323d4fcdd15abde068607cad150dfe17f7d32290ee128e5fea98442bd09"}, + {file = "pandas-1.1.1-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:0246c67cbaaaac8d25fed8d4cf2d8897bd858f0e540e8528a75281cee9ac516d"}, + {file = "pandas-1.1.1-cp38-cp38-win32.whl", hash = "sha256:01b1e536eb960822c5e6b58357cad8c4b492a336f4a5630bf0b598566462a578"}, + {file = "pandas-1.1.1-cp38-cp38-win_amd64.whl", hash = "sha256:57c5f6be49259cde8e6f71c2bf240a26b071569cabc04c751358495d09419e56"}, + {file = "pandas-1.1.1.tar.gz", hash = "sha256:53328284a7bb046e2e885fd1b8c078bd896d7fc4575b915d4936f54984a2ba67"}, +] +parso = [ + {file = "parso-0.8.3-py2.py3-none-any.whl", hash = "sha256:c001d4636cd3aecdaf33cbb40aebb59b094be2a74c556778ef5576c175e19e75"}, + {file = "parso-0.8.3.tar.gz", hash = "sha256:8c07be290bb59f03588915921e29e8a50002acaf2cdc5fa0e0114f91709fafa0"}, +] +pathspec = [ + {file = "pathspec-0.9.0-py2.py3-none-any.whl", hash = "sha256:7d15c4ddb0b5c802d161efc417ec1a2558ea2653c2e8ad9c19098201dc1c993a"}, + {file = "pathspec-0.9.0.tar.gz", hash = "sha256:e564499435a2673d586f6b2130bb5b95f04a3ba06f81b8f895b651a3c76aabb1"}, +] +pathtools = [ + {file = "pathtools-0.1.2.tar.gz", hash = "sha256:7c35c5421a39bb82e58018febd90e3b6e5db34c5443aaaf742b3f33d4655f1c0"}, +] +pbr = [ + {file = "pbr-5.8.0-py2.py3-none-any.whl", hash = "sha256:176e8560eaf61e127817ef93d8a844803abb27a4d4637f0ff3bb783129be2e0a"}, + {file = "pbr-5.8.0.tar.gz", hash = "sha256:672d8ebee84921862110f23fcec2acea191ef58543d34dfe9ef3d9f13c31cddf"}, +] +pexpect = [ + {file = "pexpect-4.8.0-py2.py3-none-any.whl", hash = "sha256:0b48a55dcb3c05f3329815901ea4fc1537514d6ba867a152b581d69ae3710937"}, + {file = "pexpect-4.8.0.tar.gz", hash = "sha256:fc65a43959d153d0114afe13997d439c22823a27cefceb5ff35c2178c6784c0c"}, +] +pickleshare = [ + {file = "pickleshare-0.7.5-py2.py3-none-any.whl", hash = "sha256:9649af414d74d4df115d5d718f82acb59c9d418196b7b4290ed47a12ce62df56"}, + {file = "pickleshare-0.7.5.tar.gz", hash = "sha256:87683d47965c1da65cdacaf31c8441d12b8044cdec9aca500cd78fc2c683afca"}, +] +pillow = [ + {file = "Pillow-8.4.0-cp310-cp310-macosx_10_10_universal2.whl", hash = "sha256:81f8d5c81e483a9442d72d182e1fb6dcb9723f289a57e8030811bac9ea3fef8d"}, + {file = "Pillow-8.4.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:3f97cfb1e5a392d75dd8b9fd274d205404729923840ca94ca45a0af57e13dbe6"}, + {file = "Pillow-8.4.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:eb9fc393f3c61f9054e1ed26e6fe912c7321af2f41ff49d3f83d05bacf22cc78"}, + {file = "Pillow-8.4.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d82cdb63100ef5eedb8391732375e6d05993b765f72cb34311fab92103314649"}, + {file = "Pillow-8.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:62cc1afda735a8d109007164714e73771b499768b9bb5afcbbee9d0ff374b43f"}, + {file = "Pillow-8.4.0-cp310-cp310-win32.whl", hash = "sha256:e3dacecfbeec9a33e932f00c6cd7996e62f53ad46fbe677577394aaa90ee419a"}, + {file = "Pillow-8.4.0-cp310-cp310-win_amd64.whl", hash = "sha256:620582db2a85b2df5f8a82ddeb52116560d7e5e6b055095f04ad828d1b0baa39"}, + {file = "Pillow-8.4.0-cp36-cp36m-macosx_10_10_x86_64.whl", hash = "sha256:1bc723b434fbc4ab50bb68e11e93ce5fb69866ad621e3c2c9bdb0cd70e345f55"}, + {file = "Pillow-8.4.0-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:72cbcfd54df6caf85cc35264c77ede902452d6df41166010262374155947460c"}, + {file = "Pillow-8.4.0-cp36-cp36m-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:70ad9e5c6cb9b8487280a02c0ad8a51581dcbbe8484ce058477692a27c151c0a"}, + {file = "Pillow-8.4.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:25a49dc2e2f74e65efaa32b153527fc5ac98508d502fa46e74fa4fd678ed6645"}, + {file = "Pillow-8.4.0-cp36-cp36m-win32.whl", hash = "sha256:93ce9e955cc95959df98505e4608ad98281fff037350d8c2671c9aa86bcf10a9"}, + {file = "Pillow-8.4.0-cp36-cp36m-win_amd64.whl", hash = "sha256:2e4440b8f00f504ee4b53fe30f4e381aae30b0568193be305256b1462216feff"}, + {file = "Pillow-8.4.0-cp37-cp37m-macosx_10_10_x86_64.whl", hash = "sha256:8c803ac3c28bbc53763e6825746f05cc407b20e4a69d0122e526a582e3b5e153"}, + {file = "Pillow-8.4.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c8a17b5d948f4ceeceb66384727dde11b240736fddeda54ca740b9b8b1556b29"}, + {file = "Pillow-8.4.0-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:1394a6ad5abc838c5cd8a92c5a07535648cdf6d09e8e2d6df916dfa9ea86ead8"}, + {file = "Pillow-8.4.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:792e5c12376594bfcb986ebf3855aa4b7c225754e9a9521298e460e92fb4a488"}, + {file = "Pillow-8.4.0-cp37-cp37m-win32.whl", hash = "sha256:d99ec152570e4196772e7a8e4ba5320d2d27bf22fdf11743dd882936ed64305b"}, + {file = "Pillow-8.4.0-cp37-cp37m-win_amd64.whl", hash = "sha256:7b7017b61bbcdd7f6363aeceb881e23c46583739cb69a3ab39cb384f6ec82e5b"}, + {file = "Pillow-8.4.0-cp38-cp38-macosx_10_10_x86_64.whl", hash = "sha256:d89363f02658e253dbd171f7c3716a5d340a24ee82d38aab9183f7fdf0cdca49"}, + {file = "Pillow-8.4.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:0a0956fdc5defc34462bb1c765ee88d933239f9a94bc37d132004775241a7585"}, + {file = "Pillow-8.4.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5b7bb9de00197fb4261825c15551adf7605cf14a80badf1761d61e59da347779"}, + {file = "Pillow-8.4.0-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:72b9e656e340447f827885b8d7a15fc8c4e68d410dc2297ef6787eec0f0ea409"}, + {file = "Pillow-8.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a5a4532a12314149d8b4e4ad8ff09dde7427731fcfa5917ff16d0291f13609df"}, + {file = "Pillow-8.4.0-cp38-cp38-win32.whl", hash = "sha256:82aafa8d5eb68c8463b6e9baeb4f19043bb31fefc03eb7b216b51e6a9981ae09"}, + {file = "Pillow-8.4.0-cp38-cp38-win_amd64.whl", hash = "sha256:066f3999cb3b070a95c3652712cffa1a748cd02d60ad7b4e485c3748a04d9d76"}, + {file = "Pillow-8.4.0-cp39-cp39-macosx_10_10_x86_64.whl", hash = "sha256:5503c86916d27c2e101b7f71c2ae2cddba01a2cf55b8395b0255fd33fa4d1f1a"}, + {file = "Pillow-8.4.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:4acc0985ddf39d1bc969a9220b51d94ed51695d455c228d8ac29fcdb25810e6e"}, + {file = "Pillow-8.4.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0b052a619a8bfcf26bd8b3f48f45283f9e977890263e4571f2393ed8898d331b"}, + {file = "Pillow-8.4.0-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:493cb4e415f44cd601fcec11c99836f707bb714ab03f5ed46ac25713baf0ff20"}, + {file = "Pillow-8.4.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b8831cb7332eda5dc89b21a7bce7ef6ad305548820595033a4b03cf3091235ed"}, + {file = "Pillow-8.4.0-cp39-cp39-win32.whl", hash = "sha256:5e9ac5f66616b87d4da618a20ab0a38324dbe88d8a39b55be8964eb520021e02"}, + {file = "Pillow-8.4.0-cp39-cp39-win_amd64.whl", hash = "sha256:3eb1ce5f65908556c2d8685a8f0a6e989d887ec4057326f6c22b24e8a172c66b"}, + {file = "Pillow-8.4.0-pp36-pypy36_pp73-macosx_10_10_x86_64.whl", hash = "sha256:ddc4d832a0f0b4c52fff973a0d44b6c99839a9d016fe4e6a1cb8f3eea96479c2"}, + {file = "Pillow-8.4.0-pp36-pypy36_pp73-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:9a3e5ddc44c14042f0844b8cf7d2cd455f6cc80fd7f5eefbe657292cf601d9ad"}, + {file = "Pillow-8.4.0-pp36-pypy36_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c70e94281588ef053ae8998039610dbd71bc509e4acbc77ab59d7d2937b10698"}, + {file = "Pillow-8.4.0-pp37-pypy37_pp73-macosx_10_10_x86_64.whl", hash = "sha256:3862b7256046fcd950618ed22d1d60b842e3a40a48236a5498746f21189afbbc"}, + {file = "Pillow-8.4.0-pp37-pypy37_pp73-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a4901622493f88b1a29bd30ec1a2f683782e57c3c16a2dbc7f2595ba01f639df"}, + {file = "Pillow-8.4.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:84c471a734240653a0ec91dec0996696eea227eafe72a33bd06c92697728046b"}, + {file = "Pillow-8.4.0-pp37-pypy37_pp73-win_amd64.whl", hash = "sha256:244cf3b97802c34c41905d22810846802a3329ddcb93ccc432870243211c79fc"}, + {file = "Pillow-8.4.0.tar.gz", hash = "sha256:b8e2f83c56e141920c39464b852de3719dfbfb6e3c99a2d8da0edf4fb33176ed"}, +] +pkginfo = [ + {file = "pkginfo-1.8.2-py2.py3-none-any.whl", hash = "sha256:c24c487c6a7f72c66e816ab1796b96ac6c3d14d49338293d2141664330b55ffc"}, + {file = "pkginfo-1.8.2.tar.gz", hash = "sha256:542e0d0b6750e2e21c20179803e40ab50598d8066d51097a0e382cba9eb02bff"}, +] +platformdirs = [ + {file = "platformdirs-2.4.1-py3-none-any.whl", hash = "sha256:1d7385c7db91728b83efd0ca99a5afb296cab9d0ed8313a45ed8ba17967ecfca"}, + {file = "platformdirs-2.4.1.tar.gz", hash = "sha256:440633ddfebcc36264232365d7840a970e75e1018d15b4327d11f91909045fda"}, +] +pluggy = [ + {file = "pluggy-0.13.1-py2.py3-none-any.whl", hash = "sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d"}, + {file = "pluggy-0.13.1.tar.gz", hash = "sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0"}, +] +pre-commit = [ + {file = "pre_commit-2.16.0-py2.py3-none-any.whl", hash = "sha256:758d1dc9b62c2ed8881585c254976d66eae0889919ab9b859064fc2fe3c7743e"}, + {file = "pre_commit-2.16.0.tar.gz", hash = "sha256:fe9897cac830aa7164dbd02a4e7b90cae49630451ce88464bca73db486ba9f65"}, +] +prompt-toolkit = [ + {file = "prompt_toolkit-3.0.24-py3-none-any.whl", hash = "sha256:e56f2ff799bacecd3e88165b1e2f5ebf9bcd59e80e06d395fa0cc4b8bd7bb506"}, + {file = "prompt_toolkit-3.0.24.tar.gz", hash = "sha256:1bb05628c7d87b645974a1bad3f17612be0c29fa39af9f7688030163f680bad6"}, +] +protobuf = [ + {file = "protobuf-3.19.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:d80f80eb175bf5f1169139c2e0c5ada98b1c098e2b3c3736667f28cbbea39fc8"}, + {file = "protobuf-3.19.1-cp310-cp310-manylinux2014_aarch64.whl", hash = "sha256:a529e7df52204565bcd33738a7a5f288f3d2d37d86caa5d78c458fa5fabbd54d"}, + {file = "protobuf-3.19.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:28ccea56d4dc38d35cd70c43c2da2f40ac0be0a355ef882242e8586c6d66666f"}, + {file = "protobuf-3.19.1-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:8b30a7de128c46b5ecb343917d9fa737612a6e8280f440874e5cc2ba0d79b8f6"}, + {file = "protobuf-3.19.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5935c8ce02e3d89c7900140a8a42b35bc037ec07a6aeb61cc108be8d3c9438a6"}, + {file = "protobuf-3.19.1-cp36-cp36m-win32.whl", hash = "sha256:74f33edeb4f3b7ed13d567881da8e5a92a72b36495d57d696c2ea1ae0cfee80c"}, + {file = "protobuf-3.19.1-cp36-cp36m-win_amd64.whl", hash = "sha256:038daf4fa38a7e818dd61f51f22588d61755160a98db087a046f80d66b855942"}, + {file = "protobuf-3.19.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:8e51561d72efd5bd5c91490af1f13e32bcba8dab4643761eb7de3ce18e64a853"}, + {file = "protobuf-3.19.1-cp37-cp37m-manylinux2014_aarch64.whl", hash = "sha256:6e8ea9173403219239cdfd8d946ed101f2ab6ecc025b0fda0c6c713c35c9981d"}, + {file = "protobuf-3.19.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:db3532d9f7a6ebbe2392041350437953b6d7a792de10e629c1e4f5a6b1fe1ac6"}, + {file = "protobuf-3.19.1-cp37-cp37m-win32.whl", hash = "sha256:615b426a177780ce381ecd212edc1e0f70db8557ed72560b82096bd36b01bc04"}, + {file = "protobuf-3.19.1-cp37-cp37m-win_amd64.whl", hash = "sha256:d8919368410110633717c406ab5c97e8df5ce93020cfcf3012834f28b1fab1ea"}, + {file = "protobuf-3.19.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:71b0250b0cfb738442d60cab68abc166de43411f2a4f791d31378590bfb71bd7"}, + {file = "protobuf-3.19.1-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:3cd0458870ea7d1c58e948ac8078f6ba8a7ecc44a57e03032ed066c5bb318089"}, + {file = "protobuf-3.19.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:655264ed0d0efe47a523e2255fc1106a22f6faab7cc46cfe99b5bae085c2a13e"}, + {file = "protobuf-3.19.1-cp38-cp38-win32.whl", hash = "sha256:b691d996c6d0984947c4cf8b7ae2fe372d99b32821d0584f0b90277aa36982d3"}, + {file = "protobuf-3.19.1-cp38-cp38-win_amd64.whl", hash = "sha256:e7e8d2c20921f8da0dea277dfefc6abac05903ceac8e72839b2da519db69206b"}, + {file = "protobuf-3.19.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:fd390367fc211cc0ffcf3a9e149dfeca78fecc62adb911371db0cec5c8b7472d"}, + {file = "protobuf-3.19.1-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:d83e1ef8cb74009bebee3e61cc84b1c9cd04935b72bca0cbc83217d140424995"}, + {file = "protobuf-3.19.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:36d90676d6f426718463fe382ec6274909337ca6319d375eebd2044e6c6ac560"}, + {file = "protobuf-3.19.1-cp39-cp39-win32.whl", hash = "sha256:e7b24c11df36ee8e0c085e5b0dc560289e4b58804746fb487287dda51410f1e2"}, + {file = "protobuf-3.19.1-cp39-cp39-win_amd64.whl", hash = "sha256:77d2fadcf369b3f22859ab25bd12bb8e98fb11e05d9ff9b7cd45b711c719c002"}, + {file = "protobuf-3.19.1-py2.py3-none-any.whl", hash = "sha256:e813b1c9006b6399308e917ac5d298f345d95bb31f46f02b60cd92970a9afa17"}, + {file = "protobuf-3.19.1.tar.gz", hash = "sha256:62a8e4baa9cb9e064eb62d1002eca820857ab2138440cb4b3ea4243830f94ca7"}, +] +ptyprocess = [ + {file = "ptyprocess-0.7.0-py2.py3-none-any.whl", hash = "sha256:4b41f3967fce3af57cc7e94b888626c18bf37a083e3651ca8feeb66d492fef35"}, + {file = "ptyprocess-0.7.0.tar.gz", hash = "sha256:5c5d0a3b48ceee0b48485e0c26037c0acd7d29765ca3fbb5cb3831d347423220"}, +] +py = [ + {file = "py-1.11.0-py2.py3-none-any.whl", hash = "sha256:607c53218732647dff4acdfcd50cb62615cedf612e72d1724fb1a0cc6405b378"}, + {file = "py-1.11.0.tar.gz", hash = "sha256:51c75c4126074b472f746a24399ad32f6053d1b34b68d2fa41e558e6f4a98719"}, +] +pyasn1 = [ + {file = "pyasn1-0.4.8-py2.4.egg", hash = "sha256:fec3e9d8e36808a28efb59b489e4528c10ad0f480e57dcc32b4de5c9d8c9fdf3"}, + {file = "pyasn1-0.4.8-py2.5.egg", hash = "sha256:0458773cfe65b153891ac249bcf1b5f8f320b7c2ce462151f8fa74de8934becf"}, + {file = "pyasn1-0.4.8-py2.6.egg", hash = "sha256:5c9414dcfede6e441f7e8f81b43b34e834731003427e5b09e4e00e3172a10f00"}, + {file = "pyasn1-0.4.8-py2.7.egg", hash = "sha256:6e7545f1a61025a4e58bb336952c5061697da694db1cae97b116e9c46abcf7c8"}, + {file = "pyasn1-0.4.8-py2.py3-none-any.whl", hash = "sha256:39c7e2ec30515947ff4e87fb6f456dfc6e84857d34be479c9d4a4ba4bf46aa5d"}, + {file = "pyasn1-0.4.8-py3.1.egg", hash = "sha256:78fa6da68ed2727915c4767bb386ab32cdba863caa7dbe473eaae45f9959da86"}, + {file = "pyasn1-0.4.8-py3.2.egg", hash = "sha256:08c3c53b75eaa48d71cf8c710312316392ed40899cb34710d092e96745a358b7"}, + {file = "pyasn1-0.4.8-py3.3.egg", hash = "sha256:03840c999ba71680a131cfaee6fab142e1ed9bbd9c693e285cc6aca0d555e576"}, + {file = "pyasn1-0.4.8-py3.4.egg", hash = "sha256:7ab8a544af125fb704feadb008c99a88805126fb525280b2270bb25cc1d78a12"}, + {file = "pyasn1-0.4.8-py3.5.egg", hash = "sha256:e89bf84b5437b532b0803ba5c9a5e054d21fec423a89952a74f87fa2c9b7bce2"}, + {file = "pyasn1-0.4.8-py3.6.egg", hash = "sha256:014c0e9976956a08139dc0712ae195324a75e142284d5f87f1a87ee1b068a359"}, + {file = "pyasn1-0.4.8-py3.7.egg", hash = "sha256:99fcc3c8d804d1bc6d9a099921e39d827026409a58f2a720dcdb89374ea0c776"}, + {file = "pyasn1-0.4.8.tar.gz", hash = "sha256:aef77c9fb94a3ac588e87841208bdec464471d9871bd5050a287cc9a475cd0ba"}, +] +pyasn1-modules = [ + {file = "pyasn1-modules-0.2.8.tar.gz", hash = "sha256:905f84c712230b2c592c19470d3ca8d552de726050d1d1716282a1f6146be65e"}, + {file = "pyasn1_modules-0.2.8-py2.4.egg", hash = "sha256:0fe1b68d1e486a1ed5473f1302bd991c1611d319bba158e98b106ff86e1d7199"}, + {file = "pyasn1_modules-0.2.8-py2.5.egg", hash = "sha256:fe0644d9ab041506b62782e92b06b8c68cca799e1a9636ec398675459e031405"}, + {file = "pyasn1_modules-0.2.8-py2.6.egg", hash = "sha256:a99324196732f53093a84c4369c996713eb8c89d360a496b599fb1a9c47fc3eb"}, + {file = "pyasn1_modules-0.2.8-py2.7.egg", hash = "sha256:0845a5582f6a02bb3e1bde9ecfc4bfcae6ec3210dd270522fee602365430c3f8"}, + {file = "pyasn1_modules-0.2.8-py2.py3-none-any.whl", hash = "sha256:a50b808ffeb97cb3601dd25981f6b016cbb3d31fbf57a8b8a87428e6158d0c74"}, + {file = "pyasn1_modules-0.2.8-py3.1.egg", hash = "sha256:f39edd8c4ecaa4556e989147ebf219227e2cd2e8a43c7e7fcb1f1c18c5fd6a3d"}, + {file = "pyasn1_modules-0.2.8-py3.2.egg", hash = "sha256:b80486a6c77252ea3a3e9b1e360bc9cf28eaac41263d173c032581ad2f20fe45"}, + {file = "pyasn1_modules-0.2.8-py3.3.egg", hash = "sha256:65cebbaffc913f4fe9e4808735c95ea22d7a7775646ab690518c056784bc21b4"}, + {file = "pyasn1_modules-0.2.8-py3.4.egg", hash = "sha256:15b7c67fabc7fc240d87fb9aabf999cf82311a6d6fb2c70d00d3d0604878c811"}, + {file = "pyasn1_modules-0.2.8-py3.5.egg", hash = "sha256:426edb7a5e8879f1ec54a1864f16b882c2837bfd06eee62f2c982315ee2473ed"}, + {file = "pyasn1_modules-0.2.8-py3.6.egg", hash = "sha256:cbac4bc38d117f2a49aeedec4407d23e8866ea4ac27ff2cf7fb3e5b570df19e0"}, + {file = "pyasn1_modules-0.2.8-py3.7.egg", hash = "sha256:c29a5e5cc7a3f05926aff34e097e84f8589cd790ce0ed41b67aed6857b26aafd"}, +] +pycodestyle = [ + {file = "pycodestyle-2.6.0-py2.py3-none-any.whl", hash = "sha256:2295e7b2f6b5bd100585ebcb1f616591b652db8a741695b3d8f5d28bdc934367"}, + {file = "pycodestyle-2.6.0.tar.gz", hash = "sha256:c58a7d2815e0e8d7972bf1803331fb0152f867bd89adf8a01dfd55085434192e"}, +] +pycparser = [ + {file = "pycparser-2.21-py2.py3-none-any.whl", hash = "sha256:8ee45429555515e1f6b185e78100aea234072576aa43ab53aefcae078162fca9"}, + {file = "pycparser-2.21.tar.gz", hash = "sha256:e644fdec12f7872f86c58ff790da456218b10f863970249516d60a5eaca77206"}, +] +pydantic = [ + {file = "pydantic-1.9.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:cb23bcc093697cdea2708baae4f9ba0e972960a835af22560f6ae4e7e47d33f5"}, + {file = "pydantic-1.9.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:1d5278bd9f0eee04a44c712982343103bba63507480bfd2fc2790fa70cd64cf4"}, + {file = "pydantic-1.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ab624700dc145aa809e6f3ec93fb8e7d0f99d9023b713f6a953637429b437d37"}, + {file = "pydantic-1.9.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c8d7da6f1c1049eefb718d43d99ad73100c958a5367d30b9321b092771e96c25"}, + {file = "pydantic-1.9.0-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:3c3b035103bd4e2e4a28da9da7ef2fa47b00ee4a9cf4f1a735214c1bcd05e0f6"}, + {file = "pydantic-1.9.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:3011b975c973819883842c5ab925a4e4298dffccf7782c55ec3580ed17dc464c"}, + {file = "pydantic-1.9.0-cp310-cp310-win_amd64.whl", hash = "sha256:086254884d10d3ba16da0588604ffdc5aab3f7f09557b998373e885c690dd398"}, + {file = "pydantic-1.9.0-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:0fe476769acaa7fcddd17cadd172b156b53546ec3614a4d880e5d29ea5fbce65"}, + {file = "pydantic-1.9.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c8e9dcf1ac499679aceedac7e7ca6d8641f0193c591a2d090282aaf8e9445a46"}, + {file = "pydantic-1.9.0-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d1e4c28f30e767fd07f2ddc6f74f41f034d1dd6bc526cd59e63a82fe8bb9ef4c"}, + {file = "pydantic-1.9.0-cp36-cp36m-musllinux_1_1_i686.whl", hash = "sha256:c86229333cabaaa8c51cf971496f10318c4734cf7b641f08af0a6fbf17ca3054"}, + {file = "pydantic-1.9.0-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:c0727bda6e38144d464daec31dff936a82917f431d9c39c39c60a26567eae3ed"}, + {file = "pydantic-1.9.0-cp36-cp36m-win_amd64.whl", hash = "sha256:dee5ef83a76ac31ab0c78c10bd7d5437bfdb6358c95b91f1ba7ff7b76f9996a1"}, + {file = "pydantic-1.9.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:d9c9bdb3af48e242838f9f6e6127de9be7063aad17b32215ccc36a09c5cf1070"}, + {file = "pydantic-1.9.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2ee7e3209db1e468341ef41fe263eb655f67f5c5a76c924044314e139a1103a2"}, + {file = "pydantic-1.9.0-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0b6037175234850ffd094ca77bf60fb54b08b5b22bc85865331dd3bda7a02fa1"}, + {file = "pydantic-1.9.0-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:b2571db88c636d862b35090ccf92bf24004393f85c8870a37f42d9f23d13e032"}, + {file = "pydantic-1.9.0-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:8b5ac0f1c83d31b324e57a273da59197c83d1bb18171e512908fe5dc7278a1d6"}, + {file = "pydantic-1.9.0-cp37-cp37m-win_amd64.whl", hash = "sha256:bbbc94d0c94dd80b3340fc4f04fd4d701f4b038ebad72c39693c794fd3bc2d9d"}, + {file = "pydantic-1.9.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:e0896200b6a40197405af18828da49f067c2fa1f821491bc8f5bde241ef3f7d7"}, + {file = "pydantic-1.9.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:7bdfdadb5994b44bd5579cfa7c9b0e1b0e540c952d56f627eb227851cda9db77"}, + {file = "pydantic-1.9.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:574936363cd4b9eed8acdd6b80d0143162f2eb654d96cb3a8ee91d3e64bf4cf9"}, + {file = "pydantic-1.9.0-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c556695b699f648c58373b542534308922c46a1cda06ea47bc9ca45ef5b39ae6"}, + {file = "pydantic-1.9.0-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:f947352c3434e8b937e3aa8f96f47bdfe6d92779e44bb3f41e4c213ba6a32145"}, + {file = "pydantic-1.9.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:5e48ef4a8b8c066c4a31409d91d7ca372a774d0212da2787c0d32f8045b1e034"}, + {file = "pydantic-1.9.0-cp38-cp38-win_amd64.whl", hash = "sha256:96f240bce182ca7fe045c76bcebfa0b0534a1bf402ed05914a6f1dadff91877f"}, + {file = "pydantic-1.9.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:815ddebb2792efd4bba5488bc8fde09c29e8ca3227d27cf1c6990fc830fd292b"}, + {file = "pydantic-1.9.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:6c5b77947b9e85a54848343928b597b4f74fc364b70926b3c4441ff52620640c"}, + {file = "pydantic-1.9.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4c68c3bc88dbda2a6805e9a142ce84782d3930f8fdd9655430d8576315ad97ce"}, + {file = "pydantic-1.9.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:5a79330f8571faf71bf93667d3ee054609816f10a259a109a0738dac983b23c3"}, + {file = "pydantic-1.9.0-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:f5a64b64ddf4c99fe201ac2724daada8595ada0d102ab96d019c1555c2d6441d"}, + {file = "pydantic-1.9.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:a733965f1a2b4090a5238d40d983dcd78f3ecea221c7af1497b845a9709c1721"}, + {file = "pydantic-1.9.0-cp39-cp39-win_amd64.whl", hash = "sha256:2cc6a4cb8a118ffec2ca5fcb47afbacb4f16d0ab8b7350ddea5e8ef7bcc53a16"}, + {file = "pydantic-1.9.0-py3-none-any.whl", hash = "sha256:085ca1de245782e9b46cefcf99deecc67d418737a1fd3f6a4f511344b613a5b3"}, + {file = "pydantic-1.9.0.tar.gz", hash = "sha256:742645059757a56ecd886faf4ed2441b9c0cd406079c2b4bee51bcc3fbcd510a"}, +] +pydocstyle = [ + {file = "pydocstyle-6.1.1-py3-none-any.whl", hash = "sha256:6987826d6775056839940041beef5c08cc7e3d71d63149b48e36727f70144dc4"}, + {file = "pydocstyle-6.1.1.tar.gz", hash = "sha256:1d41b7c459ba0ee6c345f2eb9ae827cab14a7533a88c5c6f7e94923f72df92dc"}, +] +pyflakes = [ + {file = "pyflakes-2.2.0-py2.py3-none-any.whl", hash = "sha256:0d94e0e05a19e57a99444b6ddcf9a6eb2e5c68d3ca1e98e90707af8152c90a92"}, + {file = "pyflakes-2.2.0.tar.gz", hash = "sha256:35b2d75ee967ea93b55750aa9edbbf72813e06a66ba54438df2cfac9e3c27fc8"}, +] +pygments = [ + {file = "Pygments-2.11.1-py3-none-any.whl", hash = "sha256:9135c1af61eec0f650cd1ea1ed8ce298e54d56bcd8cc2ef46edd7702c171337c"}, + {file = "Pygments-2.11.1.tar.gz", hash = "sha256:59b895e326f0fb0d733fd28c6839bd18ad0687ba20efc26d4277fd1d30b971f4"}, +] +pylint = [ + {file = "pylint-2.12.2-py3-none-any.whl", hash = "sha256:daabda3f7ed9d1c60f52d563b1b854632fd90035bcf01443e234d3dc794e3b74"}, + {file = "pylint-2.12.2.tar.gz", hash = "sha256:9d945a73640e1fec07ee34b42f5669b770c759acd536ec7b16d7e4b87a9c9ff9"}, +] +pyparsing = [ + {file = "pyparsing-3.0.6-py3-none-any.whl", hash = "sha256:04ff808a5b90911829c55c4e26f75fa5ca8a2f5f36aa3a51f68e27033341d3e4"}, + {file = "pyparsing-3.0.6.tar.gz", hash = "sha256:d9bdec0013ef1eb5a84ab39a3b3868911598afa494f5faa038647101504e2b81"}, +] +pytest = [ + {file = "pytest-6.0.1-py3-none-any.whl", hash = "sha256:8b6007800c53fdacd5a5c192203f4e531eb2a1540ad9c752e052ec0f7143dbad"}, + {file = "pytest-6.0.1.tar.gz", hash = "sha256:85228d75db9f45e06e57ef9bf4429267f81ac7c0d742cc9ed63d09886a9fe6f4"}, +] +pytest-runner = [ + {file = "pytest-runner-5.2.tar.gz", hash = "sha256:96c7e73ead7b93e388c5d614770d2bae6526efd997757d3543fe17b557a0942b"}, + {file = "pytest_runner-5.2-py2.py3-none-any.whl", hash = "sha256:5534b08b133ef9a5e2c22c7886a8f8508c95bb0b0bdc6cc13214f269c3c70d51"}, +] +python-dateutil = [ + {file = "python-dateutil-2.8.2.tar.gz", hash = "sha256:0123cacc1627ae19ddf3c27a5de5bd67ee4586fbdd6440d9748f8abb483d3e86"}, + {file = "python_dateutil-2.8.2-py2.py3-none-any.whl", hash = "sha256:961d03dc3453ebbc59dbdea9e4e11c5651520a876d0f4db161e8674aae935da9"}, +] +pytz = [ + {file = "pytz-2021.3-py2.py3-none-any.whl", hash = "sha256:3672058bc3453457b622aab7a1c3bfd5ab0bdae451512f6cf25f64ed37f5b87c"}, + {file = "pytz-2021.3.tar.gz", hash = "sha256:acad2d8b20a1af07d4e4c9d2e9285c5ed9104354062f275f3fcd88dcef4f1326"}, +] +pyupgrade = [ + {file = "pyupgrade-2.31.0-py2.py3-none-any.whl", hash = "sha256:0a62c5055f854d7f36e155b7ee8920561bf0399c53edd975cf02436eef8937fc"}, + {file = "pyupgrade-2.31.0.tar.gz", hash = "sha256:80e2308cae2b11c3fdd091137495d99abf7e0cd98b501aa5758974991497c24c"}, +] +pywin32-ctypes = [ + {file = "pywin32-ctypes-0.2.0.tar.gz", hash = "sha256:24ffc3b341d457d48e8922352130cf2644024a4ff09762a2261fd34c36ee5942"}, + {file = "pywin32_ctypes-0.2.0-py2.py3-none-any.whl", hash = "sha256:9dc2d991b3479cc2df15930958b674a48a227d5361d413827a4cfd0b5876fc98"}, +] +pyyaml = [ + {file = "PyYAML-5.3.1-cp27-cp27m-win32.whl", hash = "sha256:74809a57b329d6cc0fdccee6318f44b9b8649961fa73144a98735b0aaf029f1f"}, + {file = "PyYAML-5.3.1-cp27-cp27m-win_amd64.whl", hash = "sha256:240097ff019d7c70a4922b6869d8a86407758333f02203e0fc6ff79c5dcede76"}, + {file = "PyYAML-5.3.1-cp35-cp35m-win32.whl", hash = "sha256:4f4b913ca1a7319b33cfb1369e91e50354d6f07a135f3b901aca02aa95940bd2"}, + {file = "PyYAML-5.3.1-cp35-cp35m-win_amd64.whl", hash = "sha256:cc8955cfbfc7a115fa81d85284ee61147059a753344bc51098f3ccd69b0d7e0c"}, + {file = "PyYAML-5.3.1-cp36-cp36m-win32.whl", hash = "sha256:7739fc0fa8205b3ee8808aea45e968bc90082c10aef6ea95e855e10abf4a37b2"}, + {file = "PyYAML-5.3.1-cp36-cp36m-win_amd64.whl", hash = "sha256:69f00dca373f240f842b2931fb2c7e14ddbacd1397d57157a9b005a6a9942648"}, + {file = "PyYAML-5.3.1-cp37-cp37m-win32.whl", hash = "sha256:d13155f591e6fcc1ec3b30685d50bf0711574e2c0dfffd7644babf8b5102ca1a"}, + {file = "PyYAML-5.3.1-cp37-cp37m-win_amd64.whl", hash = "sha256:73f099454b799e05e5ab51423c7bcf361c58d3206fa7b0d555426b1f4d9a3eaf"}, + {file = "PyYAML-5.3.1-cp38-cp38-win32.whl", hash = "sha256:06a0d7ba600ce0b2d2fe2e78453a470b5a6e000a985dd4a4e54e436cc36b0e97"}, + {file = "PyYAML-5.3.1-cp38-cp38-win_amd64.whl", hash = "sha256:95f71d2af0ff4227885f7a6605c37fd53d3a106fcab511b8860ecca9fcf400ee"}, + {file = "PyYAML-5.3.1-cp39-cp39-win32.whl", hash = "sha256:ad9c67312c84def58f3c04504727ca879cb0013b2517c85a9a253f0cb6380c0a"}, + {file = "PyYAML-5.3.1-cp39-cp39-win_amd64.whl", hash = "sha256:6034f55dab5fea9e53f436aa68fa3ace2634918e8b5994d82f3621c04ff5ed2e"}, + {file = "PyYAML-5.3.1.tar.gz", hash = "sha256:b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d"}, +] +readme-renderer = [ + {file = "readme_renderer-32.0-py3-none-any.whl", hash = "sha256:a50a0f2123a4c1145ac6f420e1a348aafefcc9211c846e3d51df05fe3d865b7d"}, + {file = "readme_renderer-32.0.tar.gz", hash = "sha256:b512beafa6798260c7d5af3e1b1f097e58bfcd9a575da7c4ddd5e037490a5b85"}, +] +requests = [ + {file = "requests-2.27.0-py2.py3-none-any.whl", hash = "sha256:f71a09d7feba4a6b64ffd8e9d9bc60f9bf7d7e19fd0e04362acb1cfc2e3d98df"}, + {file = "requests-2.27.0.tar.gz", hash = "sha256:8e5643905bf20a308e25e4c1dd379117c09000bf8a82ebccc462cfb1b34a16b5"}, +] +requests-oauthlib = [ + {file = "requests-oauthlib-1.3.0.tar.gz", hash = "sha256:b4261601a71fd721a8bd6d7aa1cc1d6a8a93b4a9f5e96626f8e4d91e8beeaa6a"}, + {file = "requests_oauthlib-1.3.0-py2.py3-none-any.whl", hash = "sha256:7f71572defaecd16372f9006f33c2ec8c077c3cfa6f5911a9a90202beb513f3d"}, + {file = "requests_oauthlib-1.3.0-py3.7.egg", hash = "sha256:fa6c47b933f01060936d87ae9327fead68768b69c6c9ea2109c48be30f2d4dbc"}, +] +requests-toolbelt = [ + {file = "requests-toolbelt-0.9.1.tar.gz", hash = "sha256:968089d4584ad4ad7c171454f0a5c6dac23971e9472521ea3b6d49d610aa6fc0"}, + {file = "requests_toolbelt-0.9.1-py2.py3-none-any.whl", hash = "sha256:380606e1d10dc85c3bd47bf5a6095f815ec007be7a8b69c878507068df059e6f"}, +] +rfc3986 = [ + {file = "rfc3986-1.5.0-py2.py3-none-any.whl", hash = "sha256:a86d6e1f5b1dc238b218b012df0aa79409667bb209e58da56d0b94704e712a97"}, + {file = "rfc3986-1.5.0.tar.gz", hash = "sha256:270aaf10d87d0d4e095063c65bf3ddbc6ee3d0b226328ce21e036f946e421835"}, +] +rsa = [ + {file = "rsa-4.8-py3-none-any.whl", hash = "sha256:95c5d300c4e879ee69708c428ba566c59478fd653cc3a22243eeb8ed846950bb"}, + {file = "rsa-4.8.tar.gz", hash = "sha256:5c6bd9dc7a543b7fe4304a631f8a8a3b674e2bbfc49c2ae96200cdbe55df6b17"}, +] +safety = [ + {file = "safety-1.10.3-py2.py3-none-any.whl", hash = "sha256:5f802ad5df5614f9622d8d71fedec2757099705c2356f862847c58c6dfe13e84"}, + {file = "safety-1.10.3.tar.gz", hash = "sha256:30e394d02a20ac49b7f65292d19d38fa927a8f9582cdfd3ad1adbbc66c641ad5"}, +] +scikit-learn = [ + {file = "scikit-learn-0.23.2.tar.gz", hash = "sha256:20766f515e6cd6f954554387dfae705d93c7b544ec0e6c6a5d8e006f6f7ef480"}, + {file = "scikit_learn-0.23.2-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:98508723f44c61896a4e15894b2016762a55555fbf09365a0bb1870ecbd442de"}, + {file = "scikit_learn-0.23.2-cp36-cp36m-manylinux1_i686.whl", hash = "sha256:a64817b050efd50f9abcfd311870073e500ae11b299683a519fbb52d85e08d25"}, + {file = "scikit_learn-0.23.2-cp36-cp36m-manylinux1_x86_64.whl", hash = "sha256:daf276c465c38ef736a79bd79fc80a249f746bcbcae50c40945428f7ece074f8"}, + {file = "scikit_learn-0.23.2-cp36-cp36m-win32.whl", hash = "sha256:cb3e76380312e1f86abd20340ab1d5b3cc46a26f6593d3c33c9ea3e4c7134028"}, + {file = "scikit_learn-0.23.2-cp36-cp36m-win_amd64.whl", hash = "sha256:0a127cc70990d4c15b1019680bfedc7fec6c23d14d3719fdf9b64b22d37cdeca"}, + {file = "scikit_learn-0.23.2-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:2aa95c2f17d2f80534156215c87bee72b6aa314a7f8b8fe92a2d71f47280570d"}, + {file = "scikit_learn-0.23.2-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:6c28a1d00aae7c3c9568f61aafeaad813f0f01c729bee4fd9479e2132b215c1d"}, + {file = "scikit_learn-0.23.2-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:da8e7c302003dd765d92a5616678e591f347460ac7b53e53d667be7dfe6d1b10"}, + {file = "scikit_learn-0.23.2-cp37-cp37m-win32.whl", hash = "sha256:d9a1ce5f099f29c7c33181cc4386660e0ba891b21a60dc036bf369e3a3ee3aec"}, + {file = "scikit_learn-0.23.2-cp37-cp37m-win_amd64.whl", hash = "sha256:914ac2b45a058d3f1338d7736200f7f3b094857758895f8667be8a81ff443b5b"}, + {file = "scikit_learn-0.23.2-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:7671bbeddd7f4f9a6968f3b5442dac5f22bf1ba06709ef888cc9132ad354a9ab"}, + {file = "scikit_learn-0.23.2-cp38-cp38-manylinux1_i686.whl", hash = "sha256:d0dcaa54263307075cb93d0bee3ceb02821093b1b3d25f66021987d305d01dce"}, + {file = "scikit_learn-0.23.2-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:5ce7a8021c9defc2b75620571b350acc4a7d9763c25b7593621ef50f3bd019a2"}, + {file = "scikit_learn-0.23.2-cp38-cp38-win32.whl", hash = "sha256:0d39748e7c9669ba648acf40fb3ce96b8a07b240db6888563a7cb76e05e0d9cc"}, + {file = "scikit_learn-0.23.2-cp38-cp38-win_amd64.whl", hash = "sha256:1b8a391de95f6285a2f9adffb7db0892718950954b7149a70c783dc848f104ea"}, +] +scipy = [ + {file = "scipy-1.6.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:a15a1f3fc0abff33e792d6049161b7795909b40b97c6cc2934ed54384017ab76"}, + {file = "scipy-1.6.1-cp37-cp37m-manylinux1_i686.whl", hash = "sha256:e79570979ccdc3d165456dd62041d9556fb9733b86b4b6d818af7a0afc15f092"}, + {file = "scipy-1.6.1-cp37-cp37m-manylinux1_x86_64.whl", hash = "sha256:a423533c55fec61456dedee7b6ee7dce0bb6bfa395424ea374d25afa262be261"}, + {file = "scipy-1.6.1-cp37-cp37m-manylinux2014_aarch64.whl", hash = "sha256:33d6b7df40d197bdd3049d64e8e680227151673465e5d85723b3b8f6b15a6ced"}, + {file = "scipy-1.6.1-cp37-cp37m-win32.whl", hash = "sha256:6725e3fbb47da428794f243864f2297462e9ee448297c93ed1dcbc44335feb78"}, + {file = "scipy-1.6.1-cp37-cp37m-win_amd64.whl", hash = "sha256:5fa9c6530b1661f1370bcd332a1e62ca7881785cc0f80c0d559b636567fab63c"}, + {file = "scipy-1.6.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:bd50daf727f7c195e26f27467c85ce653d41df4358a25b32434a50d8870fc519"}, + {file = "scipy-1.6.1-cp38-cp38-manylinux1_i686.whl", hash = "sha256:f46dd15335e8a320b0fb4685f58b7471702234cba8bb3442b69a3e1dc329c345"}, + {file = "scipy-1.6.1-cp38-cp38-manylinux1_x86_64.whl", hash = "sha256:0e5b0ccf63155d90da576edd2768b66fb276446c371b73841e3503be1d63fb5d"}, + {file = "scipy-1.6.1-cp38-cp38-manylinux2014_aarch64.whl", hash = "sha256:2481efbb3740977e3c831edfd0bd9867be26387cacf24eb5e366a6a374d3d00d"}, + {file = "scipy-1.6.1-cp38-cp38-win32.whl", hash = "sha256:68cb4c424112cd4be886b4d979c5497fba190714085f46b8ae67a5e4416c32b4"}, + {file = "scipy-1.6.1-cp38-cp38-win_amd64.whl", hash = "sha256:5f331eeed0297232d2e6eea51b54e8278ed8bb10b099f69c44e2558c090d06bf"}, + {file = "scipy-1.6.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:0c8a51d33556bf70367452d4d601d1742c0e806cd0194785914daf19775f0e67"}, + {file = "scipy-1.6.1-cp39-cp39-manylinux1_i686.whl", hash = "sha256:83bf7c16245c15bc58ee76c5418e46ea1811edcc2e2b03041b804e46084ab627"}, + {file = "scipy-1.6.1-cp39-cp39-manylinux1_x86_64.whl", hash = "sha256:794e768cc5f779736593046c9714e0f3a5940bc6dcc1dba885ad64cbfb28e9f0"}, + {file = "scipy-1.6.1-cp39-cp39-manylinux2014_aarch64.whl", hash = "sha256:5da5471aed911fe7e52b86bf9ea32fb55ae93e2f0fac66c32e58897cfb02fa07"}, + {file = "scipy-1.6.1-cp39-cp39-win32.whl", hash = "sha256:8e403a337749ed40af60e537cc4d4c03febddcc56cd26e774c9b1b600a70d3e4"}, + {file = "scipy-1.6.1-cp39-cp39-win_amd64.whl", hash = "sha256:a5193a098ae9f29af283dcf0041f762601faf2e595c0db1da929875b7570353f"}, + {file = "scipy-1.6.1.tar.gz", hash = "sha256:c4fceb864890b6168e79b0e714c585dbe2fd4222768ee90bc1aa0f8218691b11"}, +] +secretstorage = [ + {file = "SecretStorage-3.3.1-py3-none-any.whl", hash = "sha256:422d82c36172d88d6a0ed5afdec956514b189ddbfb72fefab0c8a1cee4eaf71f"}, + {file = "SecretStorage-3.3.1.tar.gz", hash = "sha256:fd666c51a6bf200643495a04abb261f83229dcb6fd8472ec393df7ffc8b6f195"}, +] +six = [ + {file = "six-1.16.0-py2.py3-none-any.whl", hash = "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254"}, + {file = "six-1.16.0.tar.gz", hash = "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926"}, +] +skl2onnx = [ + {file = "skl2onnx-1.10.3-py2.py3-none-any.whl", hash = "sha256:908bccb2974b6ef852878b28a2a5e65cfe59c7572ea285aee46c64a4b6d2728a"}, + {file = "skl2onnx-1.10.3.tar.gz", hash = "sha256:798933378145412b9876ab3ff2c1dd5f241a7296406d786262000afa8d329628"}, +] +smmap = [ + {file = "smmap-5.0.0-py3-none-any.whl", hash = "sha256:2aba19d6a040e78d8b09de5c57e96207b09ed71d8e55ce0959eeee6c8e190d94"}, + {file = "smmap-5.0.0.tar.gz", hash = "sha256:c840e62059cd3be204b0c9c9f74be2c09d5648eddd4580d9314c3ecde0b30936"}, +] +snowballstemmer = [ + {file = "snowballstemmer-2.2.0-py2.py3-none-any.whl", hash = "sha256:c8e1716e83cc398ae16824e5572ae04e0d9fc2c6b985fb0f900f5f0c96ecba1a"}, + {file = "snowballstemmer-2.2.0.tar.gz", hash = "sha256:09b16deb8547d3412ad7b590689584cd0fe25ec8db3be37788be3810cbf19cb1"}, +] +sphinx = [ + {file = "Sphinx-3.2.1-py3-none-any.whl", hash = "sha256:ce6fd7ff5b215af39e2fcd44d4a321f6694b4530b6f2b2109b64d120773faea0"}, + {file = "Sphinx-3.2.1.tar.gz", hash = "sha256:321d6d9b16fa381a5306e5a0b76cd48ffbc588e6340059a729c6fdd66087e0e8"}, +] +sphinxcontrib-applehelp = [ + {file = "sphinxcontrib-applehelp-1.0.2.tar.gz", hash = "sha256:a072735ec80e7675e3f432fcae8610ecf509c5f1869d17e2eecff44389cdbc58"}, + {file = "sphinxcontrib_applehelp-1.0.2-py2.py3-none-any.whl", hash = "sha256:806111e5e962be97c29ec4c1e7fe277bfd19e9652fb1a4392105b43e01af885a"}, +] +sphinxcontrib-devhelp = [ + {file = "sphinxcontrib-devhelp-1.0.2.tar.gz", hash = "sha256:ff7f1afa7b9642e7060379360a67e9c41e8f3121f2ce9164266f61b9f4b338e4"}, + {file = "sphinxcontrib_devhelp-1.0.2-py2.py3-none-any.whl", hash = "sha256:8165223f9a335cc1af7ffe1ed31d2871f325254c0423bc0c4c7cd1c1e4734a2e"}, +] +sphinxcontrib-htmlhelp = [ + {file = "sphinxcontrib-htmlhelp-2.0.0.tar.gz", hash = "sha256:f5f8bb2d0d629f398bf47d0d69c07bc13b65f75a81ad9e2f71a63d4b7a2f6db2"}, + {file = "sphinxcontrib_htmlhelp-2.0.0-py2.py3-none-any.whl", hash = "sha256:d412243dfb797ae3ec2b59eca0e52dac12e75a241bf0e4eb861e450d06c6ed07"}, +] +sphinxcontrib-jsmath = [ + {file = "sphinxcontrib-jsmath-1.0.1.tar.gz", hash = "sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8"}, + {file = "sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl", hash = "sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178"}, +] +sphinxcontrib-qthelp = [ + {file = "sphinxcontrib-qthelp-1.0.3.tar.gz", hash = "sha256:4c33767ee058b70dba89a6fc5c1892c0d57a54be67ddd3e7875a18d14cba5a72"}, + {file = "sphinxcontrib_qthelp-1.0.3-py2.py3-none-any.whl", hash = "sha256:bd9fc24bcb748a8d51fd4ecaade681350aa63009a347a8c14e637895444dfab6"}, +] +sphinxcontrib-serializinghtml = [ + {file = "sphinxcontrib-serializinghtml-1.1.5.tar.gz", hash = "sha256:aa5f6de5dfdf809ef505c4895e51ef5c9eac17d0f287933eb49ec495280b6952"}, + {file = "sphinxcontrib_serializinghtml-1.1.5-py2.py3-none-any.whl", hash = "sha256:352a9a00ae864471d3a7ead8d7d79f5fc0b57e8b3f95e9867eb9eb28999b92fd"}, +] +starlette = [ + {file = "starlette-0.14.2-py3-none-any.whl", hash = "sha256:3c8e48e52736b3161e34c9f0e8153b4f32ec5d8995a3ee1d59410d92f75162ed"}, + {file = "starlette-0.14.2.tar.gz", hash = "sha256:7d49f4a27f8742262ef1470608c59ddbc66baf37c148e938c7038e6bc7a998aa"}, +] +stevedore = [ + {file = "stevedore-3.5.0-py3-none-any.whl", hash = "sha256:a547de73308fd7e90075bb4d301405bebf705292fa90a90fc3bcf9133f58616c"}, + {file = "stevedore-3.5.0.tar.gz", hash = "sha256:f40253887d8712eaa2bb0ea3830374416736dc8ec0e22f5a65092c1174c44335"}, +] +tensorboard = [ + {file = "tensorboard-2.7.0-py3-none-any.whl", hash = "sha256:239f78a4a8dff200ce585a030c787773a8c1184d5c159252f5f85bac4e3c3b38"}, +] +tensorboard-data-server = [ + {file = "tensorboard_data_server-0.6.1-py3-none-any.whl", hash = "sha256:809fe9887682d35c1f7d1f54f0f40f98bb1f771b14265b453ca051e2ce58fca7"}, + {file = "tensorboard_data_server-0.6.1-py3-none-macosx_10_9_x86_64.whl", hash = "sha256:fa8cef9be4fcae2f2363c88176638baf2da19c5ec90addb49b1cde05c95c88ee"}, + {file = "tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl", hash = "sha256:d8237580755e58eff68d1f3abefb5b1e39ae5c8b127cc40920f9c4fb33f4b98a"}, +] +tensorboard-plugin-wit = [ + {file = "tensorboard_plugin_wit-1.8.0-py3-none-any.whl", hash = "sha256:2a80d1c551d741e99b2f197bb915d8a133e24adb8da1732b840041860f91183a"}, +] +tensorflow = [ + {file = "tensorflow-2.3.4-cp36-cp36m-macosx_10_11_x86_64.whl", hash = "sha256:8e5cb20ad3cbb30a603eb27ec3e1baee9bc4fc8360f8ce2b8ebc9ee1d2592cfb"}, + {file = "tensorflow-2.3.4-cp36-cp36m-manylinux2010_x86_64.whl", hash = "sha256:b6d068cbdf53809688d8eff3fb5bea030f350ff7f507035832a743b4efa4094b"}, + {file = "tensorflow-2.3.4-cp36-cp36m-win_amd64.whl", hash = "sha256:dcc337db41a55038b2ba7b24266ff5704e36b0666263d3f906027eb9e651466d"}, + {file = "tensorflow-2.3.4-cp37-cp37m-macosx_10_11_x86_64.whl", hash = "sha256:807cf068ad5fb5ac70ff290425117df8751cd91c41a8e8884453185005ef4401"}, + {file = "tensorflow-2.3.4-cp37-cp37m-manylinux2010_x86_64.whl", hash = "sha256:f118f2fe8a116063ebfac6e67777a6b70e0179907c61d13b60fc2192005a6d8f"}, + {file = "tensorflow-2.3.4-cp37-cp37m-win_amd64.whl", hash = "sha256:3075f16868e710b244924ef1a1b31447b665e3b35ccc43717358709dc1d4b6a0"}, + {file = "tensorflow-2.3.4-cp38-cp38-macosx_10_11_x86_64.whl", hash = "sha256:7be77c2ca170d75b7b5c0abbf8f332facf33d1d9e6c500dd12117f75626c6ace"}, + {file = "tensorflow-2.3.4-cp38-cp38-manylinux2010_x86_64.whl", hash = "sha256:348090e6a86eee4b75eef5f72786da440c5fbf7b31bbd189758436025c0a23a9"}, + {file = "tensorflow-2.3.4-cp38-cp38-win_amd64.whl", hash = "sha256:48135c40a6c4a01eed131bc11b9b2e44cd0c2e78adb85f3b4bd51d730af06e8e"}, +] +tensorflow-estimator = [ + {file = "tensorflow_estimator-2.3.0-py2.py3-none-any.whl", hash = "sha256:b75e034300ccb169403cf2695adf3368da68863aeb0c14c3760064c713d5c486"}, +] +termcolor = [ + {file = "termcolor-1.1.0.tar.gz", hash = "sha256:1d6d69ce66211143803fbc56652b41d73b4a400a2891d7bf7a1cdf4c02de613b"}, +] +threadpoolctl = [ + {file = "threadpoolctl-3.0.0-py3-none-any.whl", hash = "sha256:4fade5b3b48ae4b1c30f200b28f39180371104fccc642e039e0f2435ec8cc211"}, + {file = "threadpoolctl-3.0.0.tar.gz", hash = "sha256:d03115321233d0be715f0d3a5ad1d6c065fe425ddc2d671ca8e45e9fd5d7a52a"}, +] +tokenize-rt = [ + {file = "tokenize_rt-4.2.1-py2.py3-none-any.whl", hash = "sha256:08a27fa032a81cf45e8858d0ac706004fcd523e8463415ddf1442be38e204ea8"}, + {file = "tokenize_rt-4.2.1.tar.gz", hash = "sha256:0d4f69026fed520f8a1e0103aa36c406ef4661417f20ca643f913e33531b3b94"}, +] +toml = [ + {file = "toml-0.10.2-py2.py3-none-any.whl", hash = "sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b"}, + {file = "toml-0.10.2.tar.gz", hash = "sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f"}, +] +tomli = [ + {file = "tomli-1.2.3-py3-none-any.whl", hash = "sha256:e3069e4be3ead9668e21cb9b074cd948f7b3113fd9c8bba083f48247aab8b11c"}, + {file = "tomli-1.2.3.tar.gz", hash = "sha256:05b6166bff487dc068d322585c7ea4ef78deed501cc124060e0f238e89a9231f"}, +] +tox = [ + {file = "tox-3.19.0-py2.py3-none-any.whl", hash = "sha256:3d94b6921a0b6dc90fd8128df83741f30bb41ccd6cd52d131a6a6944ca8f16e6"}, + {file = "tox-3.19.0.tar.gz", hash = "sha256:17e61a93afe5c49281fb969ab71f7a3f22d7586d1c56f9a74219910f356fe7d3"}, +] +tqdm = [ + {file = "tqdm-4.62.3-py2.py3-none-any.whl", hash = "sha256:8dd278a422499cd6b727e6ae4061c40b48fce8b76d1ccbf5d34fca9b7f925b0c"}, + {file = "tqdm-4.62.3.tar.gz", hash = "sha256:d359de7217506c9851b7869f3708d8ee53ed70a1b8edbba4dbcb47442592920d"}, +] +traitlets = [ + {file = "traitlets-5.1.1-py3-none-any.whl", hash = "sha256:2d313cc50a42cd6c277e7d7dc8d4d7fedd06a2c215f78766ae7b1a66277e0033"}, + {file = "traitlets-5.1.1.tar.gz", hash = "sha256:059f456c5a7c1c82b98c2e8c799f39c9b8128f6d0d46941ee118daace9eb70c7"}, +] +twine = [ + {file = "twine-3.2.0-py3-none-any.whl", hash = "sha256:ba9ff477b8d6de0c89dd450e70b2185da190514e91c42cc62f96850025c10472"}, + {file = "twine-3.2.0.tar.gz", hash = "sha256:34352fd52ec3b9d29837e6072d5a2a7c6fe4290e97bba46bb8d478b5c598f7ab"}, +] +typing-extensions = [ + {file = "typing_extensions-4.0.1-py3-none-any.whl", hash = "sha256:7f001e5ac290a0c0401508864c7ec868be4e701886d5b573a9528ed3973d9d3b"}, + {file = "typing_extensions-4.0.1.tar.gz", hash = "sha256:4ca091dea149f945ec56afb48dae714f21e8692ef22a395223bcd328961b6a0e"}, +] +urllib3 = [ + {file = "urllib3-1.26.7-py2.py3-none-any.whl", hash = "sha256:c4fdf4019605b6e5423637e01bc9fe4daef873709a7973e195ceba0a62bbc844"}, + {file = "urllib3-1.26.7.tar.gz", hash = "sha256:4987c65554f7a2dbf30c18fd48778ef124af6fab771a377103da0585e2336ece"}, +] +uvicorn = [ + {file = "uvicorn-0.14.0-py3-none-any.whl", hash = "sha256:2a76bb359171a504b3d1c853409af3adbfa5cef374a4a59e5881945a97a93eae"}, + {file = "uvicorn-0.14.0.tar.gz", hash = "sha256:45ad7dfaaa7d55cab4cd1e85e03f27e9d60bc067ddc59db52a2b0aeca8870292"}, +] +virtualenv = [ + {file = "virtualenv-20.13.0-py2.py3-none-any.whl", hash = "sha256:339f16c4a86b44240ba7223d0f93a7887c3ca04b5f9c8129da7958447d079b09"}, + {file = "virtualenv-20.13.0.tar.gz", hash = "sha256:d8458cf8d59d0ea495ad9b34c2599487f8a7772d796f9910858376d1600dd2dd"}, +] +watchdog = [ + {file = "watchdog-0.10.3.tar.gz", hash = "sha256:4214e1379d128b0588021880ccaf40317ee156d4603ac388b9adcf29165e0c04"}, +] +wcwidth = [ + {file = "wcwidth-0.2.5-py2.py3-none-any.whl", hash = "sha256:beb4802a9cebb9144e99086eff703a642a13d6a0052920003a230f3294bbe784"}, + {file = "wcwidth-0.2.5.tar.gz", hash = "sha256:c4d647b99872929fdb7bdcaa4fbe7f01413ed3d98077df798530e5b04f116c83"}, +] +webencodings = [ + {file = "webencodings-0.5.1-py2.py3-none-any.whl", hash = "sha256:a0af1213f3c2226497a97e2b3aa01a7e4bee4f403f95be16fc9acd2947514a78"}, + {file = "webencodings-0.5.1.tar.gz", hash = "sha256:b36a1c245f2d304965eb4e0a82848379241dc04b865afcc4aab16748587e1923"}, +] +werkzeug = [ + {file = "Werkzeug-2.0.2-py3-none-any.whl", hash = "sha256:63d3dc1cf60e7b7e35e97fa9861f7397283b75d765afcaefd993d6046899de8f"}, + {file = "Werkzeug-2.0.2.tar.gz", hash = "sha256:aa2bb6fc8dee8d6c504c0ac1e7f5f7dc5810a9903e793b6f715a9f015bdadb9a"}, +] +wrapt = [ + {file = "wrapt-1.13.3-cp27-cp27m-macosx_10_9_x86_64.whl", hash = "sha256:e05e60ff3b2b0342153be4d1b597bbcfd8330890056b9619f4ad6b8d5c96a81a"}, + {file = "wrapt-1.13.3-cp27-cp27m-manylinux1_i686.whl", hash = "sha256:85148f4225287b6a0665eef08a178c15097366d46b210574a658c1ff5b377489"}, + {file = "wrapt-1.13.3-cp27-cp27m-manylinux1_x86_64.whl", hash = "sha256:2dded5496e8f1592ec27079b28b6ad2a1ef0b9296d270f77b8e4a3a796cf6909"}, + {file = "wrapt-1.13.3-cp27-cp27m-manylinux2010_i686.whl", hash = "sha256:e94b7d9deaa4cc7bac9198a58a7240aaf87fe56c6277ee25fa5b3aa1edebd229"}, + {file = "wrapt-1.13.3-cp27-cp27m-manylinux2010_x86_64.whl", hash = "sha256:498e6217523111d07cd67e87a791f5e9ee769f9241fcf8a379696e25806965af"}, + {file = "wrapt-1.13.3-cp27-cp27mu-manylinux1_i686.whl", hash = "sha256:ec7e20258ecc5174029a0f391e1b948bf2906cd64c198a9b8b281b811cbc04de"}, + {file = "wrapt-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl", hash = "sha256:87883690cae293541e08ba2da22cacaae0a092e0ed56bbba8d018cc486fbafbb"}, + {file = "wrapt-1.13.3-cp27-cp27mu-manylinux2010_i686.whl", hash = "sha256:f99c0489258086308aad4ae57da9e8ecf9e1f3f30fa35d5e170b4d4896554d80"}, + {file = "wrapt-1.13.3-cp27-cp27mu-manylinux2010_x86_64.whl", hash = "sha256:6a03d9917aee887690aa3f1747ce634e610f6db6f6b332b35c2dd89412912bca"}, + {file = "wrapt-1.13.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:936503cb0a6ed28dbfa87e8fcd0a56458822144e9d11a49ccee6d9a8adb2ac44"}, + {file = "wrapt-1.13.3-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:f9c51d9af9abb899bd34ace878fbec8bf357b3194a10c4e8e0a25512826ef056"}, + {file = "wrapt-1.13.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:220a869982ea9023e163ba915077816ca439489de6d2c09089b219f4e11b6785"}, + {file = "wrapt-1.13.3-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:0877fe981fd76b183711d767500e6b3111378ed2043c145e21816ee589d91096"}, + {file = "wrapt-1.13.3-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:43e69ffe47e3609a6aec0fe723001c60c65305784d964f5007d5b4fb1bc6bf33"}, + {file = "wrapt-1.13.3-cp310-cp310-win32.whl", hash = "sha256:78dea98c81915bbf510eb6a3c9c24915e4660302937b9ae05a0947164248020f"}, + {file = "wrapt-1.13.3-cp310-cp310-win_amd64.whl", hash = "sha256:ea3e746e29d4000cd98d572f3ee2a6050a4f784bb536f4ac1f035987fc1ed83e"}, + {file = "wrapt-1.13.3-cp35-cp35m-manylinux1_i686.whl", hash = "sha256:8c73c1a2ec7c98d7eaded149f6d225a692caa1bd7b2401a14125446e9e90410d"}, + {file = "wrapt-1.13.3-cp35-cp35m-manylinux1_x86_64.whl", hash = "sha256:086218a72ec7d986a3eddb7707c8c4526d677c7b35e355875a0fe2918b059179"}, + {file = "wrapt-1.13.3-cp35-cp35m-manylinux2010_i686.whl", hash = "sha256:e92d0d4fa68ea0c02d39f1e2f9cb5bc4b4a71e8c442207433d8db47ee79d7aa3"}, + {file = "wrapt-1.13.3-cp35-cp35m-manylinux2010_x86_64.whl", hash = "sha256:d4a5f6146cfa5c7ba0134249665acd322a70d1ea61732723c7d3e8cc0fa80755"}, + {file = "wrapt-1.13.3-cp35-cp35m-win32.whl", hash = "sha256:8aab36778fa9bba1a8f06a4919556f9f8c7b33102bd71b3ab307bb3fecb21851"}, + {file = "wrapt-1.13.3-cp35-cp35m-win_amd64.whl", hash = "sha256:944b180f61f5e36c0634d3202ba8509b986b5fbaf57db3e94df11abee244ba13"}, + {file = "wrapt-1.13.3-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:2ebdde19cd3c8cdf8df3fc165bc7827334bc4e353465048b36f7deeae8ee0918"}, + {file = "wrapt-1.13.3-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:610f5f83dd1e0ad40254c306f4764fcdc846641f120c3cf424ff57a19d5f7ade"}, + {file = "wrapt-1.13.3-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:5601f44a0f38fed36cc07db004f0eedeaadbdcec90e4e90509480e7e6060a5bc"}, + {file = "wrapt-1.13.3-cp36-cp36m-musllinux_1_1_i686.whl", hash = "sha256:e6906d6f48437dfd80464f7d7af1740eadc572b9f7a4301e7dd3d65db285cacf"}, + {file = "wrapt-1.13.3-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:766b32c762e07e26f50d8a3468e3b4228b3736c805018e4b0ec8cc01ecd88125"}, + {file = "wrapt-1.13.3-cp36-cp36m-win32.whl", hash = "sha256:5f223101f21cfd41deec8ce3889dc59f88a59b409db028c469c9b20cfeefbe36"}, + {file = "wrapt-1.13.3-cp36-cp36m-win_amd64.whl", hash = "sha256:f122ccd12fdc69628786d0c947bdd9cb2733be8f800d88b5a37c57f1f1d73c10"}, + {file = "wrapt-1.13.3-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:46f7f3af321a573fc0c3586612db4decb7eb37172af1bc6173d81f5b66c2e068"}, + {file = "wrapt-1.13.3-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:778fd096ee96890c10ce96187c76b3e99b2da44e08c9e24d5652f356873f6709"}, + {file = "wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:0cb23d36ed03bf46b894cfec777eec754146d68429c30431c99ef28482b5c1df"}, + {file = "wrapt-1.13.3-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:96b81ae75591a795d8c90edc0bfaab44d3d41ffc1aae4d994c5aa21d9b8e19a2"}, + {file = "wrapt-1.13.3-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:7dd215e4e8514004c8d810a73e342c536547038fb130205ec4bba9f5de35d45b"}, + {file = "wrapt-1.13.3-cp37-cp37m-win32.whl", hash = "sha256:47f0a183743e7f71f29e4e21574ad3fa95676136f45b91afcf83f6a050914829"}, + {file = "wrapt-1.13.3-cp37-cp37m-win_amd64.whl", hash = "sha256:fd76c47f20984b43d93de9a82011bb6e5f8325df6c9ed4d8310029a55fa361ea"}, + {file = "wrapt-1.13.3-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:b73d4b78807bd299b38e4598b8e7bd34ed55d480160d2e7fdaabd9931afa65f9"}, + {file = "wrapt-1.13.3-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:ec9465dd69d5657b5d2fa6133b3e1e989ae27d29471a672416fd729b429eb554"}, + {file = "wrapt-1.13.3-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:dd91006848eb55af2159375134d724032a2d1d13bcc6f81cd8d3ed9f2b8e846c"}, + {file = "wrapt-1.13.3-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:ae9de71eb60940e58207f8e71fe113c639da42adb02fb2bcbcaccc1ccecd092b"}, + {file = "wrapt-1.13.3-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:51799ca950cfee9396a87f4a1240622ac38973b6df5ef7a41e7f0b98797099ce"}, + {file = "wrapt-1.13.3-cp38-cp38-win32.whl", hash = "sha256:4b9c458732450ec42578b5642ac53e312092acf8c0bfce140ada5ca1ac556f79"}, + {file = "wrapt-1.13.3-cp38-cp38-win_amd64.whl", hash = "sha256:7dde79d007cd6dfa65afe404766057c2409316135cb892be4b1c768e3f3a11cb"}, + {file = "wrapt-1.13.3-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:981da26722bebb9247a0601e2922cedf8bb7a600e89c852d063313102de6f2cb"}, + {file = "wrapt-1.13.3-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:705e2af1f7be4707e49ced9153f8d72131090e52be9278b5dbb1498c749a1e32"}, + {file = "wrapt-1.13.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:25b1b1d5df495d82be1c9d2fad408f7ce5ca8a38085e2da41bb63c914baadff7"}, + {file = "wrapt-1.13.3-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:77416e6b17926d953b5c666a3cb718d5945df63ecf922af0ee576206d7033b5e"}, + {file = "wrapt-1.13.3-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:865c0b50003616f05858b22174c40ffc27a38e67359fa1495605f96125f76640"}, + {file = "wrapt-1.13.3-cp39-cp39-win32.whl", hash = "sha256:0a017a667d1f7411816e4bf214646d0ad5b1da2c1ea13dec6c162736ff25a374"}, + {file = "wrapt-1.13.3-cp39-cp39-win_amd64.whl", hash = "sha256:81bd7c90d28a4b2e1df135bfbd7c23aee3050078ca6441bead44c42483f9ebfb"}, + {file = "wrapt-1.13.3.tar.gz", hash = "sha256:1fea9cd438686e6682271d36f3481a9f3636195578bab9ca3382e2f5f01fc185"}, +] +zipp = [ + {file = "zipp-3.7.0-py3-none-any.whl", hash = "sha256:b47250dd24f92b7dd6a0a8fc5244da14608f3ca90a5efcd37a3b1642fac9a375"}, + {file = "zipp-3.7.0.tar.gz", hash = "sha256:9f50f446828eb9d45b267433fd3e9da8d801f614129124863f9c51ebceafb87d"}, +] diff --git a/igel/pyproject.toml b/igel/pyproject.toml new file mode 100644 index 0000000..7ee694a --- /dev/null +++ b/igel/pyproject.toml @@ -0,0 +1,78 @@ +[tool.poetry] +name = "igel" +version = "0.1.0" +readme = "docs/README.rst" +description = "a delightful machine learning tool that allows you to train, test and use models without writing code" +authors = ["nidhal baccouri "] +license = "MIT" + +# Pypi classifiers: https://pypi.org/classifiers/ +classifiers = [ # Update me + "Development Status :: 4 - Beta", + "Intended Audience :: Developers", + "Operating System :: OS Independent", + "Topic :: Software Development :: Libraries :: Python Modules", +] + +[tool.poetry.dependencies] +python = ">=3.8" +importlib_metadata = {version = ">=1.6.0", python = "<3.10"} +pandas = "1.1.1" +PyYAML = "5.3.1" +scikit-learn = "0.23.2" +joblib = "0.16.0" +fastapi = "~0.65.0" +uvicorn = "~0.14.0" +click = "~8.0.1" +Pillow = "^8.3.2" +autokeras = "^1.0.16" +skl2onnx = "^1.10.3" +tqdm = "^4.66.1" + +[tool.poetry.dev-dependencies] +pip="20.2.2" +bump2version="1.0.0" +wheel="0.35.1" +watchdog="0.10.3" +flake8="3.8.3" +tox="3.19.0" +coverage="5.2.1" +Sphinx="3.2.1" +twine="3.2.0" +pytest="6.0.1" +pytest-runner="5.2" + +darglint = "^1.5.8" +isort = "^5.7.0" +pyupgrade = "^2.19.3" +black = "^21.5b2" +mypy = "^0.902" +bandit = "^1.7.0" +safety = "^1.10.3" +pylint = "^2.6.0" +pydocstyle = "^6.1.1" +pre-commit = "^2.9.3" + +[build-system] +requires = ["poetry-core>=1.0.0"] +build-backend = "poetry.core.masonry.api" + +[tool.poetry.scripts] +"igel" = "igel.__main__:cli" + +[tool.black] +# https://github.com/psf/black +line-length = 80 +target-version = ["py37"] + +[tool.isort] +# https://github.com/timothycrosley/isort/ +known_typing = "typing,types,typing_extensions,mypy,mypy_extensions" +sections = "FUTURE,TYPING,STDLIB,THIRDPARTY,FIRSTPARTY,LOCALFOLDER" +include_trailing_comma = true +default_section = "FIRSTPARTY" +multi_line_output = 3 +indent = 4 +force_grid_wrap = 0 +use_parentheses = true +line_length = 80 diff --git a/igel/pytest.ini b/igel/pytest.ini new file mode 100644 index 0000000..b805e58 --- /dev/null +++ b/igel/pytest.ini @@ -0,0 +1,4 @@ +[pytest] +addopts = -p no:warnings +filterwarnings = + ignore::DeprecationWarning diff --git a/igel/scripts/check_docstrings.py b/igel/scripts/check_docstrings.py new file mode 100644 index 0000000..454f5fb --- /dev/null +++ b/igel/scripts/check_docstrings.py @@ -0,0 +1,20 @@ +import subprocess +import sys + +def main(): + # You can change 'igel' to the directory you want to check + result = subprocess.run( + ["pydocstyle", "igel"], + capture_output=True, + text=True + ) + if result.returncode == 0: + print("All docstrings are present and correct! 🎉") + sys.exit(0) + else: + print("Missing or incorrect docstrings found:\n") + print(result.stdout) + sys.exit(result.returncode) + +if __name__ == "__main__": + main() diff --git a/igel/scripts/cleanup.py b/igel/scripts/cleanup.py new file mode 100644 index 0000000..75e0b1d --- /dev/null +++ b/igel/scripts/cleanup.py @@ -0,0 +1,34 @@ +import shutil +import os + +# List of paths to remove +paths = [ + "build", + "dist", + ".pytest_cache", + ".mypy_cache", + ".tox", + ".coverage", + "coverage.xml", + "htmlcov", + "*.egg-info", + "__pycache__", + ".cache", + ".venv", + "venv", +] + +def remove_path(path): + if os.path.isdir(path): + print(f"Removing directory: {path}") + shutil.rmtree(path, ignore_errors=True) + elif os.path.isfile(path): + print(f"Removing file: {path}") + os.remove(path) + +for path in paths: + # Handle wildcards + for match in [p for p in os.listdir('.') if p.startswith(path.replace("*", ""))]: + remove_path(match) + # Try to remove the path directly + remove_path(path) diff --git a/igel/setup.cfg b/igel/setup.cfg new file mode 100644 index 0000000..b09b038 --- /dev/null +++ b/igel/setup.cfg @@ -0,0 +1,28 @@ +[bumpversion] +current_version = 0.7.0 + +[darglint] +strictness = long +docstring_style = google + +[mypy] +python_version = 3.7 +pretty = True +allow_redefinition = False +check_untyped_defs = True +disallow_any_generics = True +disallow_incomplete_defs = True +ignore_missing_imports = True +implicit_reexport = False +strict_optional = True +strict_equality = True +no_implicit_optional = True +warn_no_return = True +warn_unused_ignores = True +warn_redundant_casts = True +warn_unused_configs = True +warn_return_any = True +warn_unreachable = True +show_error_codes = True +show_column_numbers = True +show_error_context = True diff --git a/igel/synthetic_data.py b/igel/synthetic_data.py new file mode 100644 index 0000000..2984431 --- /dev/null +++ b/igel/synthetic_data.py @@ -0,0 +1,94 @@ +""" +Synthetic Data Generation Utilities + +- Tabular data generation +- Time series data generation +- Data with specific distributions +""" + +import numpy as np +import pandas as pd +from datetime import datetime, timedelta + +def generate_tabular_data(n_samples=1000, n_features=10, target_column=True): + """ + Generate synthetic tabular data for classification/regression. + + Args: + n_samples: Number of samples to generate + n_features: Number of features + target_column: Whether to include a target column + + Returns: + DataFrame with synthetic data + """ + # Generate features + features = np.random.randn(n_samples, n_features) + feature_names = [f'feature_{i}' for i in range(n_features)] + + # Create DataFrame + df = pd.DataFrame(features, columns=feature_names) + + # Add target column if requested + if target_column: + # Simple target based on first feature + df['target'] = (df['feature_0'] > 0).astype(int) + + return df + +def generate_time_series_data(n_samples=1000, trend=True, seasonality=True): + """ + Generate synthetic time series data. + + Args: + n_samples: Number of time points + trend: Whether to add trend + seasonality: Whether to add seasonality + + Returns: + DataFrame with time series data + """ + # Base time series + time_index = pd.date_range(start='2020-01-01', periods=n_samples, freq='D') + + # Generate base signal + signal = np.random.randn(n_samples) * 0.1 + + # Add trend + if trend: + trend_component = np.linspace(0, 2, n_samples) + signal += trend_component + + # Add seasonality + if seasonality: + seasonal_component = 0.5 * np.sin(2 * np.pi * np.arange(n_samples) / 365) + signal += seasonal_component + + # Create DataFrame + df = pd.DataFrame({ + 'date': time_index, + 'value': signal + }) + + return df + +def generate_categorical_data(n_samples=1000, n_categories=5): + """ + Generate synthetic categorical data. + + Args: + n_samples: Number of samples + n_categories: Number of categories + + Returns: + DataFrame with categorical data + """ + categories = [f'category_{i}' for i in range(n_categories)] + data = np.random.choice(categories, size=n_samples) + + df = pd.DataFrame({ + 'category': data, + 'value': np.random.randn(n_samples) + }) + + return df \ No newline at end of file diff --git a/igel/tests/__init__.py b/igel/tests/__init__.py new file mode 100644 index 0000000..22d55e0 --- /dev/null +++ b/igel/tests/__init__.py @@ -0,0 +1 @@ +"""Unit test package for igel.""" diff --git a/igel/tests/test_igel/__init__.py b/igel/tests/test_igel/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/igel/tests/test_igel/constants.py b/igel/tests/test_igel/constants.py new file mode 100644 index 0000000..9fde63d --- /dev/null +++ b/igel/tests/test_igel/constants.py @@ -0,0 +1,24 @@ +import os +from pathlib import Path + + +class Constants: + train_data_file = "train_data.csv" + eval_data_file = "eval_data.csv" + test_data_file = "new_data.csv" + igel_yaml_file = "igel.yaml" + model_results_file = "model_results" + cwd = Path(os.path.dirname(__file__)) + model_results_dir = cwd / model_results_file + description_file = model_results_dir / "description.json" + evaluation_file = model_results_dir / "evaluation.json" + prediction_file = model_results_dir / "prediction.csv" + model_file = model_results_dir / "model.joblib" + onnx_model_file = model_results_dir / "model.onnx" + + data_dir = cwd / "data" + igel_files = cwd / "igel_files" + train_data = data_dir / train_data_file + eval_data = data_dir / eval_data_file + test_data = data_dir / test_data_file + yaml_file = igel_files / igel_yaml_file diff --git a/igel/tests/test_igel/data/eval_data.csv b/igel/tests/test_igel/data/eval_data.csv new file mode 100644 index 0000000..97a1d28 --- /dev/null +++ b/igel/tests/test_igel/data/eval_data.csv @@ -0,0 +1,100 @@ +n_pregnant,plasma_concentration,blood_pressure,TST,insulin,BMI,DPF,age,sick +6,148,72,35,0,33.6,627,50,1 +1,85,66,29,0,26.6,351,31,0 +8,183,64,0,0,23.3,672,32,1 +1,89,66,23,94,28.1,167,21,0 +0,137,40,35,168,43.1,2288,33,1 +5,116,74,0,0,25.6,201,30,0 +3,78,50,32,88,31.0,248,26,1 +10,115,0,0,0,35.3,134,29,0 +2,197,70,45,543,30.5,158,53,1 +8,125,96,0,0,0.0,232,54,1 +4,110,92,0,0,37.6,191,30,0 +10,168,74,0,0,38.0,537,34,1 +10,139,80,0,0,27.1,1441,57,0 +1,189,60,23,846,30.1,398,59,1 +5,166,72,19,175,25.8,587,51,1 +7,100,0,0,0,30.0,484,32,1 +0,118,84,47,230,45.8,551,31,1 +7,107,74,0,0,29.6,254,31,1 +1,103,30,38,83,43.3,183,33,0 +1,115,70,30,96,34.6,529,32,1 +3,126,88,41,235,39.3,704,27,0 +8,99,84,0,0,35.4,388,50,0 +7,196,90,0,0,39.8,451,41,1 +9,119,80,35,0,29.0,263,29,1 +11,143,94,33,146,36.6,254,51,1 +10,125,70,26,115,31.1,205,41,1 +7,147,76,0,0,39.4,257,43,1 +1,97,66,15,140,23.2,487,22,0 +13,145,82,19,110,22.2,245,57,0 +5,117,92,0,0,34.1,337,38,0 +5,109,75,26,0,36.0,546,60,0 +3,158,76,36,245,31.6,851,28,1 +3,88,58,11,54,24.8,267,22,0 +6,92,92,0,0,19.9,188,28,0 +10,122,78,31,0,27.6,512,45,0 +4,103,60,33,192,24.0,966,33,0 +11,138,76,0,0,33.2,420,35,0 +9,102,76,37,0,32.9,665,46,1 +2,90,68,42,0,38.2,503,27,1 +4,111,72,47,207,37.1,1390,56,1 +3,180,64,25,70,34.0,271,26,0 +7,133,84,0,0,40.2,696,37,0 +7,106,92,18,0,22.7,235,48,0 +9,171,110,24,240,45.4,721,54,1 +7,159,64,0,0,27.4,294,40,0 +0,180,66,39,0,42.0,1893,25,1 +1,146,56,0,0,29.7,564,29,0 +2,71,70,27,0,28.0,586,22,0 +7,103,66,32,0,39.1,344,31,1 +7,105,0,0,0,0.0,305,24,0 +1,103,80,11,82,19.4,491,22,0 +1,101,50,15,36,24.2,526,26,0 +5,88,66,21,23,24.4,342,30,0 +8,176,90,34,300,33.7,467,58,1 +7,150,66,42,342,34.7,718,42,0 +1,73,50,10,0,23.0,248,21,0 +7,187,68,39,304,37.7,254,41,1 +0,100,88,60,110,46.8,962,31,0 +0,146,82,0,0,40.5,1781,44,0 +0,105,64,41,142,41.5,173,22,0 +2,84,0,0,0,0.0,304,21,0 +8,133,72,0,0,32.9,270,39,1 +5,44,62,0,0,25.0,587,36,0 +2,141,58,34,128,25.4,699,24,0 +7,114,66,0,0,32.8,258,42,1 +5,99,74,27,0,29.0,203,32,0 +0,109,88,30,0,32.5,855,38,1 +2,109,92,0,0,42.7,845,54,0 +1,95,66,13,38,19.6,334,25,0 +4,146,85,27,100,28.9,189,27,0 +2,100,66,20,90,32.9,867,28,1 +5,139,64,35,140,28.6,411,26,0 +13,126,90,0,0,43.4,583,42,1 +4,129,86,20,270,35.1,231,23,0 +1,79,75,30,0,32.0,396,22,0 +1,0,48,20,0,24.7,140,22,0 +7,62,78,0,0,32.6,391,41,0 +5,95,72,33,0,37.7,370,27,0 +0,131,0,0,0,43.2,270,26,1 +2,112,66,22,0,25.0,307,24,0 +3,113,44,13,0,22.4,140,22,0 +2,74,0,0,0,0.0,102,22,0 +7,83,78,26,71,29.3,767,36,0 +0,101,65,28,0,24.6,237,22,0 +5,137,108,0,0,48.8,227,37,1 +2,110,74,29,125,32.4,698,27,0 +13,106,72,54,0,36.6,178,45,0 +2,100,68,25,71,38.5,324,26,0 +15,136,70,32,110,37.1,153,43,1 +1,107,68,19,0,26.5,165,24,0 +1,80,55,0,0,19.1,258,21,0 +4,123,80,15,176,32.0,443,34,0 +7,81,78,40,48,46.7,261,42,0 +4,134,72,0,0,23.8,277,60,1 +2,142,82,18,64,24.7,761,21,0 +6,144,72,27,228,33.9,255,40,0 +2,92,62,28,0,31.6,130,24,0 +1,71,48,18,76,20.4,323,22,0 +6,93,50,30,64,28.7,356,23,0 diff --git a/igel/tests/test_igel/data/new_data.csv b/igel/tests/test_igel/data/new_data.csv new file mode 100644 index 0000000..edb3648 --- /dev/null +++ b/igel/tests/test_igel/data/new_data.csv @@ -0,0 +1,22 @@ +n_pregnant,plasma_concentration,blood_pressure,TST,insulin,BMI,DPF,age +6,148,72,35,0,33.6,627,50 +1,85,66,29,0,26.6,351,31 +8,183,64,0,0,23.3,672,32 +1,89,66,23,94,28.1,167,21 +0,137,40,35,168,43.1,2288,33 +5,116,74,0,0,25.6,201,30 +3,78,50,32,88,31.0,248,26 +10,115,0,0,0,35.3,134,29 +2,197,70,45,543,30.5,158,53 +8,125,96,0,0,0.0,232,54 +4,110,92,0,0,37.6,191,30 +10,168,74,0,0,38.0,537,34 +10,139,80,0,0,27.1,1441,57 +1,189,60,23,846,30.1,398,59 +5,166,72,19,175,25.8,587,51 +7,100,0,0,0,30.0,484,32 +0,118,84,47,230,45.8,551,31 +7,107,74,0,0,29.6,254,31 +1,103,30,38,83,43.3,183,33 +1,115,70,30,96,34.6,529,32 +3,126,88,41,235,39.3,704,27 diff --git a/igel/tests/test_igel/data/train_data.csv b/igel/tests/test_igel/data/train_data.csv new file mode 100644 index 0000000..7602de4 --- /dev/null +++ b/igel/tests/test_igel/data/train_data.csv @@ -0,0 +1,769 @@ +n_pregnant,plasma_concentration,blood_pressure,TST,insulin,BMI,DPF,age,sick +6,148,72,35,0,33.6,627,50,1 +1,85,66,29,0,26.6,351,31,0 +8,183,64,0,0,23.3,672,32,1 +1,89,66,23,94,28.1,167,21,0 +0,137,40,35,168,43.1,2288,33,1 +5,116,74,0,0,25.6,201,30,0 +3,78,50,32,88,31.0,248,26,1 +10,115,0,0,0,35.3,134,29,0 +2,197,70,45,543,30.5,158,53,1 +8,125,96,0,0,0.0,232,54,1 +4,110,92,0,0,37.6,191,30,0 +10,168,74,0,0,38.0,537,34,1 +10,139,80,0,0,27.1,1441,57,0 +1,189,60,23,846,30.1,398,59,1 +5,166,72,19,175,25.8,587,51,1 +7,100,0,0,0,30.0,484,32,1 +0,118,84,47,230,45.8,551,31,1 +7,107,74,0,0,29.6,254,31,1 +1,103,30,38,83,43.3,183,33,0 +1,115,70,30,96,34.6,529,32,1 +3,126,88,41,235,39.3,704,27,0 +8,99,84,0,0,35.4,388,50,0 +7,196,90,0,0,39.8,451,41,1 +9,119,80,35,0,29.0,263,29,1 +11,143,94,33,146,36.6,254,51,1 +10,125,70,26,115,31.1,205,41,1 +7,147,76,0,0,39.4,257,43,1 +1,97,66,15,140,23.2,487,22,0 +13,145,82,19,110,22.2,245,57,0 +5,117,92,0,0,34.1,337,38,0 +5,109,75,26,0,36.0,546,60,0 +3,158,76,36,245,31.6,851,28,1 +3,88,58,11,54,24.8,267,22,0 +6,92,92,0,0,19.9,188,28,0 +10,122,78,31,0,27.6,512,45,0 +4,103,60,33,192,24.0,966,33,0 +11,138,76,0,0,33.2,420,35,0 +9,102,76,37,0,32.9,665,46,1 +2,90,68,42,0,38.2,503,27,1 +4,111,72,47,207,37.1,1390,56,1 +3,180,64,25,70,34.0,271,26,0 +7,133,84,0,0,40.2,696,37,0 +7,106,92,18,0,22.7,235,48,0 +9,171,110,24,240,45.4,721,54,1 +7,159,64,0,0,27.4,294,40,0 +0,180,66,39,0,42.0,1893,25,1 +1,146,56,0,0,29.7,564,29,0 +2,71,70,27,0,28.0,586,22,0 +7,103,66,32,0,39.1,344,31,1 +7,105,0,0,0,0.0,305,24,0 +1,103,80,11,82,19.4,491,22,0 +1,101,50,15,36,24.2,526,26,0 +5,88,66,21,23,24.4,342,30,0 +8,176,90,34,300,33.7,467,58,1 +7,150,66,42,342,34.7,718,42,0 +1,73,50,10,0,23.0,248,21,0 +7,187,68,39,304,37.7,254,41,1 +0,100,88,60,110,46.8,962,31,0 +0,146,82,0,0,40.5,1781,44,0 +0,105,64,41,142,41.5,173,22,0 +2,84,0,0,0,0.0,304,21,0 +8,133,72,0,0,32.9,270,39,1 +5,44,62,0,0,25.0,587,36,0 +2,141,58,34,128,25.4,699,24,0 +7,114,66,0,0,32.8,258,42,1 +5,99,74,27,0,29.0,203,32,0 +0,109,88,30,0,32.5,855,38,1 +2,109,92,0,0,42.7,845,54,0 +1,95,66,13,38,19.6,334,25,0 +4,146,85,27,100,28.9,189,27,0 +2,100,66,20,90,32.9,867,28,1 +5,139,64,35,140,28.6,411,26,0 +13,126,90,0,0,43.4,583,42,1 +4,129,86,20,270,35.1,231,23,0 +1,79,75,30,0,32.0,396,22,0 +1,0,48,20,0,24.7,140,22,0 +7,62,78,0,0,32.6,391,41,0 +5,95,72,33,0,37.7,370,27,0 +0,131,0,0,0,43.2,270,26,1 +2,112,66,22,0,25.0,307,24,0 +3,113,44,13,0,22.4,140,22,0 +2,74,0,0,0,0.0,102,22,0 +7,83,78,26,71,29.3,767,36,0 +0,101,65,28,0,24.6,237,22,0 +5,137,108,0,0,48.8,227,37,1 +2,110,74,29,125,32.4,698,27,0 +13,106,72,54,0,36.6,178,45,0 +2,100,68,25,71,38.5,324,26,0 +15,136,70,32,110,37.1,153,43,1 +1,107,68,19,0,26.5,165,24,0 +1,80,55,0,0,19.1,258,21,0 +4,123,80,15,176,32.0,443,34,0 +7,81,78,40,48,46.7,261,42,0 +4,134,72,0,0,23.8,277,60,1 +2,142,82,18,64,24.7,761,21,0 +6,144,72,27,228,33.9,255,40,0 +2,92,62,28,0,31.6,130,24,0 +1,71,48,18,76,20.4,323,22,0 +6,93,50,30,64,28.7,356,23,0 +1,122,90,51,220,49.7,325,31,1 +1,163,72,0,0,39.0,1222,33,1 +1,151,60,0,0,26.1,179,22,0 +0,125,96,0,0,22.5,262,21,0 +1,81,72,18,40,26.6,283,24,0 +2,85,65,0,0,39.6,930,27,0 +1,126,56,29,152,28.7,801,21,0 +1,96,122,0,0,22.4,207,27,0 +4,144,58,28,140,29.5,287,37,0 +3,83,58,31,18,34.3,336,25,0 +0,95,85,25,36,37.4,247,24,1 +3,171,72,33,135,33.3,199,24,1 +8,155,62,26,495,34.0,543,46,1 +1,89,76,34,37,31.2,192,23,0 +4,76,62,0,0,34.0,391,25,0 +7,160,54,32,175,30.5,588,39,1 +4,146,92,0,0,31.2,539,61,1 +5,124,74,0,0,34.0,220,38,1 +5,78,48,0,0,33.7,654,25,0 +4,97,60,23,0,28.2,443,22,0 +4,99,76,15,51,23.2,223,21,0 +0,162,76,56,100,53.2,759,25,1 +6,111,64,39,0,34.2,260,24,0 +2,107,74,30,100,33.6,404,23,0 +5,132,80,0,0,26.8,186,69,0 +0,113,76,0,0,33.3,278,23,1 +1,88,30,42,99,55.0,496,26,1 +3,120,70,30,135,42.9,452,30,0 +1,118,58,36,94,33.3,261,23,0 +1,117,88,24,145,34.5,403,40,1 +0,105,84,0,0,27.9,741,62,1 +4,173,70,14,168,29.7,361,33,1 +9,122,56,0,0,33.3,1114,33,1 +3,170,64,37,225,34.5,356,30,1 +8,84,74,31,0,38.3,457,39,0 +2,96,68,13,49,21.1,647,26,0 +2,125,60,20,140,33.8,88,31,0 +0,100,70,26,50,30.8,597,21,0 +0,93,60,25,92,28.7,532,22,0 +0,129,80,0,0,31.2,703,29,0 +5,105,72,29,325,36.9,159,28,0 +3,128,78,0,0,21.1,268,55,0 +5,106,82,30,0,39.5,286,38,0 +2,108,52,26,63,32.5,318,22,0 +10,108,66,0,0,32.4,272,42,1 +4,154,62,31,284,32.8,237,23,0 +0,102,75,23,0,0.0,572,21,0 +9,57,80,37,0,32.8,96,41,0 +2,106,64,35,119,30.5,1400,34,0 +5,147,78,0,0,33.7,218,65,0 +2,90,70,17,0,27.3,85,22,0 +1,136,74,50,204,37.4,399,24,0 +4,114,65,0,0,21.9,432,37,0 +9,156,86,28,155,34.3,1189,42,1 +1,153,82,42,485,40.6,687,23,0 +8,188,78,0,0,47.9,137,43,1 +7,152,88,44,0,50.0,337,36,1 +2,99,52,15,94,24.6,637,21,0 +1,109,56,21,135,25.2,833,23,0 +2,88,74,19,53,29.0,229,22,0 +17,163,72,41,114,40.9,817,47,1 +4,151,90,38,0,29.7,294,36,0 +7,102,74,40,105,37.2,204,45,0 +0,114,80,34,285,44.2,167,27,0 +2,100,64,23,0,29.7,368,21,0 +0,131,88,0,0,31.6,743,32,1 +6,104,74,18,156,29.9,722,41,1 +3,148,66,25,0,32.5,256,22,0 +4,120,68,0,0,29.6,709,34,0 +4,110,66,0,0,31.9,471,29,0 +3,111,90,12,78,28.4,495,29,0 +6,102,82,0,0,30.8,180,36,1 +6,134,70,23,130,35.4,542,29,1 +2,87,0,23,0,28.9,773,25,0 +1,79,60,42,48,43.5,678,23,0 +2,75,64,24,55,29.7,370,33,0 +8,179,72,42,130,32.7,719,36,1 +6,85,78,0,0,31.2,382,42,0 +0,129,110,46,130,67.1,319,26,1 +5,143,78,0,0,45.0,190,47,0 +5,130,82,0,0,39.1,956,37,1 +6,87,80,0,0,23.2,84,32,0 +0,119,64,18,92,34.9,725,23,0 +1,0,74,20,23,27.7,299,21,0 +5,73,60,0,0,26.8,268,27,0 +4,141,74,0,0,27.6,244,40,0 +7,194,68,28,0,35.9,745,41,1 +8,181,68,36,495,30.1,615,60,1 +1,128,98,41,58,32.0,1321,33,1 +8,109,76,39,114,27.9,640,31,1 +5,139,80,35,160,31.6,361,25,1 +3,111,62,0,0,22.6,142,21,0 +9,123,70,44,94,33.1,374,40,0 +7,159,66,0,0,30.4,383,36,1 +11,135,0,0,0,52.3,578,40,1 +8,85,55,20,0,24.4,136,42,0 +5,158,84,41,210,39.4,395,29,1 +1,105,58,0,0,24.3,187,21,0 +3,107,62,13,48,22.9,678,23,1 +4,109,64,44,99,34.8,905,26,1 +4,148,60,27,318,30.9,150,29,1 +0,113,80,16,0,31.0,874,21,0 +1,138,82,0,0,40.1,236,28,0 +0,108,68,20,0,27.3,787,32,0 +2,99,70,16,44,20.4,235,27,0 +6,103,72,32,190,37.7,324,55,0 +5,111,72,28,0,23.9,407,27,0 +8,196,76,29,280,37.5,605,57,1 +5,162,104,0,0,37.7,151,52,1 +1,96,64,27,87,33.2,289,21,0 +7,184,84,33,0,35.5,355,41,1 +2,81,60,22,0,27.7,290,25,0 +0,147,85,54,0,42.8,375,24,0 +7,179,95,31,0,34.2,164,60,0 +0,140,65,26,130,42.6,431,24,1 +9,112,82,32,175,34.2,260,36,1 +12,151,70,40,271,41.8,742,38,1 +5,109,62,41,129,35.8,514,25,1 +6,125,68,30,120,30.0,464,32,0 +5,85,74,22,0,29.0,1224,32,1 +5,112,66,0,0,37.8,261,41,1 +0,177,60,29,478,34.6,1072,21,1 +2,158,90,0,0,31.6,805,66,1 +7,119,0,0,0,25.2,209,37,0 +7,142,60,33,190,28.8,687,61,0 +1,100,66,15,56,23.6,666,26,0 +1,87,78,27,32,34.6,101,22,0 +0,101,76,0,0,35.7,198,26,0 +3,162,52,38,0,37.2,652,24,1 +4,197,70,39,744,36.7,2329,31,0 +0,117,80,31,53,45.2,89,24,0 +4,142,86,0,0,44.0,645,22,1 +6,134,80,37,370,46.2,238,46,1 +1,79,80,25,37,25.4,583,22,0 +4,122,68,0,0,35.0,394,29,0 +3,74,68,28,45,29.7,293,23,0 +4,171,72,0,0,43.6,479,26,1 +7,181,84,21,192,35.9,586,51,1 +0,179,90,27,0,44.1,686,23,1 +9,164,84,21,0,30.8,831,32,1 +0,104,76,0,0,18.4,582,27,0 +1,91,64,24,0,29.2,192,21,0 +4,91,70,32,88,33.1,446,22,0 +3,139,54,0,0,25.6,402,22,1 +6,119,50,22,176,27.1,1318,33,1 +2,146,76,35,194,38.2,329,29,0 +9,184,85,15,0,30.0,1213,49,1 +10,122,68,0,0,31.2,258,41,0 +0,165,90,33,680,52.3,427,23,0 +9,124,70,33,402,35.4,282,34,0 +1,111,86,19,0,30.1,143,23,0 +9,106,52,0,0,31.2,380,42,0 +2,129,84,0,0,28.0,284,27,0 +2,90,80,14,55,24.4,249,24,0 +0,86,68,32,0,35.8,238,25,0 +12,92,62,7,258,27.6,926,44,1 +1,113,64,35,0,33.6,543,21,1 +3,111,56,39,0,30.1,557,30,0 +2,114,68,22,0,28.7,92,25,0 +1,193,50,16,375,25.9,655,24,0 +11,155,76,28,150,33.3,1353,51,1 +3,191,68,15,130,30.9,299,34,0 +3,141,0,0,0,30.0,761,27,1 +4,95,70,32,0,32.1,612,24,0 +3,142,80,15,0,32.4,200,63,0 +4,123,62,0,0,32.0,226,35,1 +5,96,74,18,67,33.6,997,43,0 +0,138,0,0,0,36.3,933,25,1 +2,128,64,42,0,40.0,1101,24,0 +0,102,52,0,0,25.1,78,21,0 +2,146,0,0,0,27.5,240,28,1 +10,101,86,37,0,45.6,1136,38,1 +2,108,62,32,56,25.2,128,21,0 +3,122,78,0,0,23.0,254,40,0 +1,71,78,50,45,33.2,422,21,0 +13,106,70,0,0,34.2,251,52,0 +2,100,70,52,57,40.5,677,25,0 +7,106,60,24,0,26.5,296,29,1 +0,104,64,23,116,27.8,454,23,0 +5,114,74,0,0,24.9,744,57,0 +2,108,62,10,278,25.3,881,22,0 +0,146,70,0,0,37.9,334,28,1 +10,129,76,28,122,35.9,280,39,0 +7,133,88,15,155,32.4,262,37,0 +7,161,86,0,0,30.4,165,47,1 +2,108,80,0,0,27.0,259,52,1 +7,136,74,26,135,26.0,647,51,0 +5,155,84,44,545,38.7,619,34,0 +1,119,86,39,220,45.6,808,29,1 +4,96,56,17,49,20.8,340,26,0 +5,108,72,43,75,36.1,263,33,0 +0,78,88,29,40,36.9,434,21,0 +0,107,62,30,74,36.6,757,25,1 +2,128,78,37,182,43.3,1224,31,1 +1,128,48,45,194,40.5,613,24,1 +0,161,50,0,0,21.9,254,65,0 +6,151,62,31,120,35.5,692,28,0 +2,146,70,38,360,28.0,337,29,1 +0,126,84,29,215,30.7,520,24,0 +14,100,78,25,184,36.6,412,46,1 +8,112,72,0,0,23.6,840,58,0 +0,167,0,0,0,32.3,839,30,1 +2,144,58,33,135,31.6,422,25,1 +5,77,82,41,42,35.8,156,35,0 +5,115,98,0,0,52.9,209,28,1 +3,150,76,0,0,21.0,207,37,0 +2,120,76,37,105,39.7,215,29,0 +10,161,68,23,132,25.5,326,47,1 +0,137,68,14,148,24.8,143,21,0 +0,128,68,19,180,30.5,1391,25,1 +2,124,68,28,205,32.9,875,30,1 +6,80,66,30,0,26.2,313,41,0 +0,106,70,37,148,39.4,605,22,0 +2,155,74,17,96,26.6,433,27,1 +3,113,50,10,85,29.5,626,25,0 +7,109,80,31,0,35.9,1127,43,1 +2,112,68,22,94,34.1,315,26,0 +3,99,80,11,64,19.3,284,30,0 +3,182,74,0,0,30.5,345,29,1 +3,115,66,39,140,38.1,150,28,0 +6,194,78,0,0,23.5,129,59,1 +4,129,60,12,231,27.5,527,31,0 +3,112,74,30,0,31.6,197,25,1 +0,124,70,20,0,27.4,254,36,1 +13,152,90,33,29,26.8,731,43,1 +2,112,75,32,0,35.7,148,21,0 +1,157,72,21,168,25.6,123,24,0 +1,122,64,32,156,35.1,692,30,1 +10,179,70,0,0,35.1,200,37,0 +2,102,86,36,120,45.5,127,23,1 +6,105,70,32,68,30.8,122,37,0 +8,118,72,19,0,23.1,1476,46,0 +2,87,58,16,52,32.7,166,25,0 +1,180,0,0,0,43.3,282,41,1 +12,106,80,0,0,23.6,137,44,0 +1,95,60,18,58,23.9,260,22,0 +0,165,76,43,255,47.9,259,26,0 +0,117,0,0,0,33.8,932,44,0 +5,115,76,0,0,31.2,343,44,1 +9,152,78,34,171,34.2,893,33,1 +7,178,84,0,0,39.9,331,41,1 +1,130,70,13,105,25.9,472,22,0 +1,95,74,21,73,25.9,673,36,0 +1,0,68,35,0,32.0,389,22,0 +5,122,86,0,0,34.7,290,33,0 +8,95,72,0,0,36.8,485,57,0 +8,126,88,36,108,38.5,349,49,0 +1,139,46,19,83,28.7,654,22,0 +3,116,0,0,0,23.5,187,23,0 +3,99,62,19,74,21.8,279,26,0 +5,0,80,32,0,41.0,346,37,1 +4,92,80,0,0,42.2,237,29,0 +4,137,84,0,0,31.2,252,30,0 +3,61,82,28,0,34.4,243,46,0 +1,90,62,12,43,27.2,580,24,0 +3,90,78,0,0,42.7,559,21,0 +9,165,88,0,0,30.4,302,49,1 +1,125,50,40,167,33.3,962,28,1 +13,129,0,30,0,39.9,569,44,1 +12,88,74,40,54,35.3,378,48,0 +1,196,76,36,249,36.5,875,29,1 +5,189,64,33,325,31.2,583,29,1 +5,158,70,0,0,29.8,207,63,0 +5,103,108,37,0,39.2,305,65,0 +4,146,78,0,0,38.5,520,67,1 +4,147,74,25,293,34.9,385,30,0 +5,99,54,28,83,34.0,499,30,0 +6,124,72,0,0,27.6,368,29,1 +0,101,64,17,0,21.0,252,21,0 +3,81,86,16,66,27.5,306,22,0 +1,133,102,28,140,32.8,234,45,1 +3,173,82,48,465,38.4,2137,25,1 +0,118,64,23,89,0.0,1731,21,0 +0,84,64,22,66,35.8,545,21,0 +2,105,58,40,94,34.9,225,25,0 +2,122,52,43,158,36.2,816,28,0 +12,140,82,43,325,39.2,528,58,1 +0,98,82,15,84,25.2,299,22,0 +1,87,60,37,75,37.2,509,22,0 +4,156,75,0,0,48.3,238,32,1 +0,93,100,39,72,43.4,1021,35,0 +1,107,72,30,82,30.8,821,24,0 +0,105,68,22,0,20.0,236,22,0 +1,109,60,8,182,25.4,947,21,0 +1,90,62,18,59,25.1,1268,25,0 +1,125,70,24,110,24.3,221,25,0 +1,119,54,13,50,22.3,205,24,0 +5,116,74,29,0,32.3,660,35,1 +8,105,100,36,0,43.3,239,45,1 +5,144,82,26,285,32.0,452,58,1 +3,100,68,23,81,31.6,949,28,0 +1,100,66,29,196,32.0,444,42,0 +5,166,76,0,0,45.7,340,27,1 +1,131,64,14,415,23.7,389,21,0 +4,116,72,12,87,22.1,463,37,0 +4,158,78,0,0,32.9,803,31,1 +2,127,58,24,275,27.7,1600,25,0 +3,96,56,34,115,24.7,944,39,0 +0,131,66,40,0,34.3,196,22,1 +3,82,70,0,0,21.1,389,25,0 +3,193,70,31,0,34.9,241,25,1 +4,95,64,0,0,32.0,161,31,1 +6,137,61,0,0,24.2,151,55,0 +5,136,84,41,88,35.0,286,35,1 +9,72,78,25,0,31.6,280,38,0 +5,168,64,0,0,32.9,135,41,1 +2,123,48,32,165,42.1,520,26,0 +4,115,72,0,0,28.9,376,46,1 +0,101,62,0,0,21.9,336,25,0 +8,197,74,0,0,25.9,1191,39,1 +1,172,68,49,579,42.4,702,28,1 +6,102,90,39,0,35.7,674,28,0 +1,112,72,30,176,34.4,528,25,0 +1,143,84,23,310,42.4,1076,22,0 +1,143,74,22,61,26.2,256,21,0 +0,138,60,35,167,34.6,534,21,1 +3,173,84,33,474,35.7,258,22,1 +1,97,68,21,0,27.2,1095,22,0 +4,144,82,32,0,38.5,554,37,1 +1,83,68,0,0,18.2,624,27,0 +3,129,64,29,115,26.4,219,28,1 +1,119,88,41,170,45.3,507,26,0 +2,94,68,18,76,26.0,561,21,0 +0,102,64,46,78,40.6,496,21,0 +2,115,64,22,0,30.8,421,21,0 +8,151,78,32,210,42.9,516,36,1 +4,184,78,39,277,37.0,264,31,1 +0,94,0,0,0,0.0,256,25,0 +1,181,64,30,180,34.1,328,38,1 +0,135,94,46,145,40.6,284,26,0 +1,95,82,25,180,35.0,233,43,1 +2,99,0,0,0,22.2,108,23,0 +3,89,74,16,85,30.4,551,38,0 +1,80,74,11,60,30.0,527,22,0 +2,139,75,0,0,25.6,167,29,0 +1,90,68,8,0,24.5,1138,36,0 +0,141,0,0,0,42.4,205,29,1 +12,140,85,33,0,37.4,244,41,0 +5,147,75,0,0,29.9,434,28,0 +1,97,70,15,0,18.2,147,21,0 +6,107,88,0,0,36.8,727,31,0 +0,189,104,25,0,34.3,435,41,1 +2,83,66,23,50,32.2,497,22,0 +4,117,64,27,120,33.2,230,24,0 +8,108,70,0,0,30.5,955,33,1 +4,117,62,12,0,29.7,380,30,1 +0,180,78,63,14,59.4,2420,25,1 +1,100,72,12,70,25.3,658,28,0 +0,95,80,45,92,36.5,330,26,0 +0,104,64,37,64,33.6,510,22,1 +0,120,74,18,63,30.5,285,26,0 +1,82,64,13,95,21.2,415,23,0 +2,134,70,0,0,28.9,542,23,1 +0,91,68,32,210,39.9,381,25,0 +2,119,0,0,0,19.6,832,72,0 +2,100,54,28,105,37.8,498,24,0 +14,175,62,30,0,33.6,212,38,1 +1,135,54,0,0,26.7,687,62,0 +5,86,68,28,71,30.2,364,24,0 +10,148,84,48,237,37.6,1001,51,1 +9,134,74,33,60,25.9,460,81,0 +9,120,72,22,56,20.8,733,48,0 +1,71,62,0,0,21.8,416,26,0 +8,74,70,40,49,35.3,705,39,0 +5,88,78,30,0,27.6,258,37,0 +10,115,98,0,0,24.0,1022,34,0 +0,124,56,13,105,21.8,452,21,0 +0,74,52,10,36,27.8,269,22,0 +0,97,64,36,100,36.8,600,25,0 +8,120,0,0,0,30.0,183,38,1 +6,154,78,41,140,46.1,571,27,0 +1,144,82,40,0,41.3,607,28,0 +0,137,70,38,0,33.2,170,22,0 +0,119,66,27,0,38.8,259,22,0 +7,136,90,0,0,29.9,210,50,0 +4,114,64,0,0,28.9,126,24,0 +0,137,84,27,0,27.3,231,59,0 +2,105,80,45,191,33.7,711,29,1 +7,114,76,17,110,23.8,466,31,0 +8,126,74,38,75,25.9,162,39,0 +4,132,86,31,0,28.0,419,63,0 +3,158,70,30,328,35.5,344,35,1 +0,123,88,37,0,35.2,197,29,0 +4,85,58,22,49,27.8,306,28,0 +0,84,82,31,125,38.2,233,23,0 +0,145,0,0,0,44.2,630,31,1 +0,135,68,42,250,42.3,365,24,1 +1,139,62,41,480,40.7,536,21,0 +0,173,78,32,265,46.5,1159,58,0 +4,99,72,17,0,25.6,294,28,0 +8,194,80,0,0,26.1,551,67,0 +2,83,65,28,66,36.8,629,24,0 +2,89,90,30,0,33.5,292,42,0 +4,99,68,38,0,32.8,145,33,0 +4,125,70,18,122,28.9,1144,45,1 +3,80,0,0,0,0.0,174,22,0 +6,166,74,0,0,26.6,304,66,0 +5,110,68,0,0,26.0,292,30,0 +2,81,72,15,76,30.1,547,25,0 +7,195,70,33,145,25.1,163,55,1 +6,154,74,32,193,29.3,839,39,0 +2,117,90,19,71,25.2,313,21,0 +3,84,72,32,0,37.2,267,28,0 +6,0,68,41,0,39.0,727,41,1 +7,94,64,25,79,33.3,738,41,0 +3,96,78,39,0,37.3,238,40,0 +10,75,82,0,0,33.3,263,38,0 +0,180,90,26,90,36.5,314,35,1 +1,130,60,23,170,28.6,692,21,0 +2,84,50,23,76,30.4,968,21,0 +8,120,78,0,0,25.0,409,64,0 +12,84,72,31,0,29.7,297,46,1 +0,139,62,17,210,22.1,207,21,0 +9,91,68,0,0,24.2,200,58,0 +2,91,62,0,0,27.3,525,22,0 +3,99,54,19,86,25.6,154,24,0 +3,163,70,18,105,31.6,268,28,1 +9,145,88,34,165,30.3,771,53,1 +7,125,86,0,0,37.6,304,51,0 +13,76,60,0,0,32.8,180,41,0 +6,129,90,7,326,19.6,582,60,0 +2,68,70,32,66,25.0,187,25,0 +3,124,80,33,130,33.2,305,26,0 +6,114,0,0,0,0.0,189,26,0 +9,130,70,0,0,34.2,652,45,1 +3,125,58,0,0,31.6,151,24,0 +3,87,60,18,0,21.8,444,21,0 +1,97,64,19,82,18.2,299,21,0 +3,116,74,15,105,26.3,107,24,0 +0,117,66,31,188,30.8,493,22,0 +0,111,65,0,0,24.6,660,31,0 +2,122,60,18,106,29.8,717,22,0 +0,107,76,0,0,45.3,686,24,0 +1,86,66,52,65,41.3,917,29,0 +6,91,0,0,0,29.8,501,31,0 +1,77,56,30,56,33.3,1251,24,0 +4,132,0,0,0,32.9,302,23,1 +0,105,90,0,0,29.6,197,46,0 +0,57,60,0,0,21.7,735,67,0 +0,127,80,37,210,36.3,804,23,0 +3,129,92,49,155,36.4,968,32,1 +8,100,74,40,215,39.4,661,43,1 +3,128,72,25,190,32.4,549,27,1 +10,90,85,32,0,34.9,825,56,1 +4,84,90,23,56,39.5,159,25,0 +1,88,78,29,76,32.0,365,29,0 +8,186,90,35,225,34.5,423,37,1 +5,187,76,27,207,43.6,1034,53,1 +4,131,68,21,166,33.1,160,28,0 +1,164,82,43,67,32.8,341,50,0 +4,189,110,31,0,28.5,680,37,0 +1,116,70,28,0,27.4,204,21,0 +3,84,68,30,106,31.9,591,25,0 +6,114,88,0,0,27.8,247,66,0 +1,88,62,24,44,29.9,422,23,0 +1,84,64,23,115,36.9,471,28,0 +7,124,70,33,215,25.5,161,37,0 +1,97,70,40,0,38.1,218,30,0 +8,110,76,0,0,27.8,237,58,0 +11,103,68,40,0,46.2,126,42,0 +11,85,74,0,0,30.1,300,35,0 +6,125,76,0,0,33.8,121,54,1 +0,198,66,32,274,41.3,502,28,1 +1,87,68,34,77,37.6,401,24,0 +6,99,60,19,54,26.9,497,32,0 +0,91,80,0,0,32.4,601,27,0 +2,95,54,14,88,26.1,748,22,0 +1,99,72,30,18,38.6,412,21,0 +6,92,62,32,126,32.0,85,46,0 +4,154,72,29,126,31.3,338,37,0 +0,121,66,30,165,34.3,203,33,1 +3,78,70,0,0,32.5,270,39,0 +2,130,96,0,0,22.6,268,21,0 +3,111,58,31,44,29.5,430,22,0 +2,98,60,17,120,34.7,198,22,0 +1,143,86,30,330,30.1,892,23,0 +1,119,44,47,63,35.5,280,25,0 +6,108,44,20,130,24.0,813,35,0 +2,118,80,0,0,42.9,693,21,1 +10,133,68,0,0,27.0,245,36,0 +2,197,70,99,0,34.7,575,62,1 +0,151,90,46,0,42.1,371,21,1 +6,109,60,27,0,25.0,206,27,0 +12,121,78,17,0,26.5,259,62,0 +8,100,76,0,0,38.7,190,42,0 +8,124,76,24,600,28.7,687,52,1 +1,93,56,11,0,22.5,417,22,0 +8,143,66,0,0,34.9,129,41,1 +6,103,66,0,0,24.3,249,29,0 +3,176,86,27,156,33.3,1154,52,1 +0,73,0,0,0,21.1,342,25,0 +11,111,84,40,0,46.8,925,45,1 +2,112,78,50,140,39.4,175,24,0 +3,132,80,0,0,34.4,402,44,1 +2,82,52,22,115,28.5,1699,25,0 +6,123,72,45,230,33.6,733,34,0 +0,188,82,14,185,32.0,682,22,1 +0,67,76,0,0,45.3,194,46,0 +1,89,24,19,25,27.8,559,21,0 +1,173,74,0,0,36.8,88,38,1 +1,109,38,18,120,23.1,407,26,0 +1,108,88,19,0,27.1,400,24,0 +6,96,0,0,0,23.7,190,28,0 +1,124,74,36,0,27.8,100,30,0 +7,150,78,29,126,35.2,692,54,1 +4,183,0,0,0,28.4,212,36,1 +1,124,60,32,0,35.8,514,21,0 +1,181,78,42,293,40.0,1258,22,1 +1,92,62,25,41,19.5,482,25,0 +0,152,82,39,272,41.5,270,27,0 +1,111,62,13,182,24.0,138,23,0 +3,106,54,21,158,30.9,292,24,0 +3,174,58,22,194,32.9,593,36,1 +7,168,88,42,321,38.2,787,40,1 +6,105,80,28,0,32.5,878,26,0 +11,138,74,26,144,36.1,557,50,1 +3,106,72,0,0,25.8,207,27,0 +6,117,96,0,0,28.7,157,30,0 +2,68,62,13,15,20.1,257,23,0 +9,112,82,24,0,28.2,1282,50,1 +0,119,0,0,0,32.4,141,24,1 +2,112,86,42,160,38.4,246,28,0 +2,92,76,20,0,24.2,1698,28,0 +6,183,94,0,0,40.8,1461,45,0 +0,94,70,27,115,43.5,347,21,0 +2,108,64,0,0,30.8,158,21,0 +4,90,88,47,54,37.7,362,29,0 +0,125,68,0,0,24.7,206,21,0 +0,132,78,0,0,32.4,393,21,0 +5,128,80,0,0,34.6,144,45,0 +4,94,65,22,0,24.7,148,21,0 +7,114,64,0,0,27.4,732,34,1 +0,102,78,40,90,34.5,238,24,0 +2,111,60,0,0,26.2,343,23,0 +1,128,82,17,183,27.5,115,22,0 +10,92,62,0,0,25.9,167,31,0 +13,104,72,0,0,31.2,465,38,1 +5,104,74,0,0,28.8,153,48,0 +2,94,76,18,66,31.6,649,23,0 +7,97,76,32,91,40.9,871,32,1 +1,100,74,12,46,19.5,149,28,0 +0,102,86,17,105,29.3,695,27,0 +4,128,70,0,0,34.3,303,24,0 +6,147,80,0,0,29.5,178,50,1 +4,90,0,0,0,28.0,610,31,0 +3,103,72,30,152,27.6,730,27,0 +2,157,74,35,440,39.4,134,30,0 +1,167,74,17,144,23.4,447,33,1 +0,179,50,36,159,37.8,455,22,1 +11,136,84,35,130,28.3,260,42,1 +0,107,60,25,0,26.4,133,23,0 +1,91,54,25,100,25.2,234,23,0 +1,117,60,23,106,33.8,466,27,0 +5,123,74,40,77,34.1,269,28,0 +2,120,54,0,0,26.8,455,27,0 +1,106,70,28,135,34.2,142,22,0 +2,155,52,27,540,38.7,240,25,1 +2,101,58,35,90,21.8,155,22,0 +1,120,80,48,200,38.9,1162,41,0 +11,127,106,0,0,39.0,190,51,0 +3,80,82,31,70,34.2,1292,27,1 +10,162,84,0,0,27.7,182,54,0 +1,199,76,43,0,42.9,1394,22,1 +8,167,106,46,231,37.6,165,43,1 +9,145,80,46,130,37.9,637,40,1 +6,115,60,39,0,33.7,245,40,1 +1,112,80,45,132,34.8,217,24,0 +4,145,82,18,0,32.5,235,70,1 +10,111,70,27,0,27.5,141,40,1 +6,98,58,33,190,34.0,430,43,0 +9,154,78,30,100,30.9,164,45,0 +6,165,68,26,168,33.6,631,49,0 +1,99,58,10,0,25.4,551,21,0 +10,68,106,23,49,35.5,285,47,0 +3,123,100,35,240,57.3,880,22,0 +8,91,82,0,0,35.6,587,68,0 +6,195,70,0,0,30.9,328,31,1 +9,156,86,0,0,24.8,230,53,1 +0,93,60,0,0,35.3,263,25,0 +3,121,52,0,0,36.0,127,25,1 +2,101,58,17,265,24.2,614,23,0 +2,56,56,28,45,24.2,332,22,0 +0,162,76,36,0,49.6,364,26,1 +0,95,64,39,105,44.6,366,22,0 +4,125,80,0,0,32.3,536,27,1 +5,136,82,0,0,0.0,640,69,0 +2,129,74,26,205,33.2,591,25,0 +3,130,64,0,0,23.1,314,22,0 +1,107,50,19,0,28.3,181,29,0 +1,140,74,26,180,24.1,828,23,0 +1,144,82,46,180,46.1,335,46,1 +8,107,80,0,0,24.6,856,34,0 +13,158,114,0,0,42.3,257,44,1 +2,121,70,32,95,39.1,886,23,0 +7,129,68,49,125,38.5,439,43,1 +2,90,60,0,0,23.5,191,25,0 +7,142,90,24,480,30.4,128,43,1 +3,169,74,19,125,29.9,268,31,1 +0,99,0,0,0,25.0,253,22,0 +4,127,88,11,155,34.5,598,28,0 +4,118,70,0,0,44.5,904,26,0 +2,122,76,27,200,35.9,483,26,0 +6,125,78,31,0,27.6,565,49,1 +1,168,88,29,0,35.0,905,52,1 +2,129,0,0,0,38.5,304,41,0 +4,110,76,20,100,28.4,118,27,0 +6,80,80,36,0,39.8,177,28,0 +10,115,0,0,0,0.0,261,30,1 +2,127,46,21,335,34.4,176,22,0 +9,164,78,0,0,32.8,148,45,1 +2,93,64,32,160,38.0,674,23,1 +3,158,64,13,387,31.2,295,24,0 +5,126,78,27,22,29.6,439,40,0 +10,129,62,36,0,41.2,441,38,1 +0,134,58,20,291,26.4,352,21,0 +3,102,74,0,0,29.5,121,32,0 +7,187,50,33,392,33.9,826,34,1 +3,173,78,39,185,33.8,970,31,1 +10,94,72,18,0,23.1,595,56,0 +1,108,60,46,178,35.5,415,24,0 +5,97,76,27,0,35.6,378,52,1 +4,83,86,19,0,29.3,317,34,0 +1,114,66,36,200,38.1,289,21,0 +1,149,68,29,127,29.3,349,42,1 +5,117,86,30,105,39.1,251,42,0 +1,111,94,0,0,32.8,265,45,0 +4,112,78,40,0,39.4,236,38,0 +1,116,78,29,180,36.1,496,25,0 +0,141,84,26,0,32.4,433,22,0 +2,175,88,0,0,22.9,326,22,0 +2,92,52,0,0,30.1,141,22,0 +3,130,78,23,79,28.4,323,34,1 +8,120,86,0,0,28.4,259,22,1 +2,174,88,37,120,44.5,646,24,1 +2,106,56,27,165,29.0,426,22,0 +2,105,75,0,0,23.3,560,53,0 +4,95,60,32,0,35.4,284,28,0 +0,126,86,27,120,27.4,515,21,0 +8,65,72,23,0,32.0,600,42,0 +2,99,60,17,160,36.6,453,21,0 +1,102,74,0,0,39.5,293,42,1 +11,120,80,37,150,42.3,785,48,1 +3,102,44,20,94,30.8,400,26,0 +1,109,58,18,116,28.5,219,22,0 +9,140,94,0,0,32.7,734,45,1 +13,153,88,37,140,40.6,1174,39,0 +12,100,84,33,105,30.0,488,46,0 +1,147,94,41,0,49.3,358,27,1 +1,81,74,41,57,46.3,1096,32,0 +3,187,70,22,200,36.4,408,36,1 +6,162,62,0,0,24.3,178,50,1 +4,136,70,0,0,31.2,1182,22,1 +1,121,78,39,74,39.0,261,28,0 +3,108,62,24,0,26.0,223,25,0 +0,181,88,44,510,43.3,222,26,1 +8,154,78,32,0,32.4,443,45,1 +1,128,88,39,110,36.5,1057,37,1 +7,137,90,41,0,32.0,391,39,0 +0,123,72,0,0,36.3,258,52,1 +1,106,76,0,0,37.5,197,26,0 +6,190,92,0,0,35.5,278,66,1 +2,88,58,26,16,28.4,766,22,0 +9,170,74,31,0,44.0,403,43,1 +9,89,62,0,0,22.5,142,33,0 +10,101,76,48,180,32.9,171,63,0 +2,122,70,27,0,36.8,340,27,0 +5,121,72,23,112,26.2,245,30,0 +1,126,60,0,0,30.1,349,47,1 +1,93,70,31,0,30.4,315,23,0 diff --git a/igel/tests/test_igel/helper.py b/igel/tests/test_igel/helper.py new file mode 100644 index 0000000..812c7d0 --- /dev/null +++ b/igel/tests/test_igel/helper.py @@ -0,0 +1,27 @@ +import logging +import os +import shutil + + +def remove_file(f: str) -> None: + """ + Remove a file at the given path if it exists. + + Args: + f (str): The path to the file to be removed. + """ + try: + if os.path.exists(f): + os.remove(f) + except Exception as ex: + print(ex) + + +def remove_folder(folder, all_content=True): + try: + if all_content: + shutil.rmtree(folder) + else: + os.rmdir(folder) + except Exception as ex: + print(ex) diff --git a/igel/tests/test_igel/igel_files/igel.yaml b/igel/tests/test_igel/igel_files/igel.yaml new file mode 100644 index 0000000..c1abac7 --- /dev/null +++ b/igel/tests/test_igel/igel_files/igel.yaml @@ -0,0 +1,32 @@ + +# dataset operations +dataset: + type: csv + random_numbers: + generate_reproducible: true + seed: 42 + split: # split options + test_size: 0.2 # 0.2 means 20% for the test data, so 80% are automatically for training + shuffle: True # whether to shuffle the data before/while splitting + + preprocess: # preprocessing options + missing_values: mean # other possible values: [drop, median, most_frequent, constant] check the docs for more + encoding: + type: oneHotEncoding # other possible values: [labelEncoding] + scale: # scaling options + method: standard # standardization will scale values to have a 0 mean and 1 standard deviation | you can also try minmax + target: inputs # scale inputs. | other possible values: [outputs, all] # if you choose all then all values in the dataset will be scaled + + +# model definition +model: + type: classification + algorithm: RandomForest + arguments: + n_estimators: 100 + max_depth: 30 + + +# target you want to predict +target: + - sick diff --git a/igel/tests/test_igel/mock.py b/igel/tests/test_igel/mock.py new file mode 100644 index 0000000..64d320d --- /dev/null +++ b/igel/tests/test_igel/mock.py @@ -0,0 +1,24 @@ +import os +from pathlib import Path + +from .constants import Constants + + +class MockCliArgs: + """mock class to provide mock cli args""" + + fit = { + "cmd": "fit", + "data_path": Constants.train_data, + "yaml_path": Constants.yaml_file, + } + evaluate = {"cmd": "evaluate", "data_path": Constants.eval_data} + predict = {"cmd": "predict", "data_path": Constants.test_data} + experiment = { + "cmd": "experiment", + "data_paths": f"{Constants.train_data} {Constants.eval_data} {Constants.test_data}", + } + export = { + "cmd" : "export", + "model_path": Constants.model_file + } diff --git a/igel/tests/test_igel/test_ab_testing.py b/igel/tests/test_igel/test_ab_testing.py new file mode 100644 index 0000000..afd6c34 --- /dev/null +++ b/igel/tests/test_igel/test_ab_testing.py @@ -0,0 +1,105 @@ +"""Tests for the A/B testing functionality.""" + +import numpy as np +from sklearn.datasets import make_classification, make_regression +from sklearn.model_selection import train_test_split +from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor +import pytest + +from igel.ab_testing import ModelComparison + +@pytest.fixture +def classification_data(): + """Generate sample classification data.""" + X, y = make_classification( + n_samples=1000, + n_features=20, + n_informative=15, + n_redundant=5, + random_state=42 + ) + return train_test_split(X, y, test_size=0.2, random_state=42) + +@pytest.fixture +def regression_data(): + """Generate sample regression data.""" + X, y = make_regression( + n_samples=1000, + n_features=20, + n_informative=15, + random_state=42 + ) + return train_test_split(X, y, test_size=0.2, random_state=42) + +def test_classification_comparison(classification_data): + """Test model comparison for classification problems.""" + X_train, X_test, y_train, y_test = classification_data + + # Create two models with different parameters + model_a = RandomForestClassifier(n_estimators=100, random_state=42) + model_b = RandomForestClassifier(n_estimators=50, random_state=42) + + # Train models + model_a.fit(X_train, y_train) + model_b.fit(X_train, y_train) + + # Compare models + comparison = ModelComparison(model_a, model_b, test_type="classification") + results = comparison.compare_predictions(X_test, y_test) + + # Basic assertions + assert "model_a_metrics" in results + assert "model_b_metrics" in results + assert "statistical_test" in results + assert "accuracy" in results["model_a_metrics"] + assert "accuracy" in results["model_b_metrics"] + assert results["model_a_metrics"]["accuracy"] > 0 + assert results["model_b_metrics"]["accuracy"] > 0 + +def test_regression_comparison(regression_data): + """Test model comparison for regression problems.""" + X_train, X_test, y_train, y_test = regression_data + + # Create two models with different parameters + model_a = RandomForestRegressor(n_estimators=100, random_state=42) + model_b = RandomForestRegressor(n_estimators=50, random_state=42) + + # Train models + model_a.fit(X_train, y_train) + model_b.fit(X_train, y_train) + + # Compare models + comparison = ModelComparison(model_a, model_b, test_type="regression") + results = comparison.compare_predictions(X_test, y_test) + + # Basic assertions + assert "model_a_metrics" in results + assert "model_b_metrics" in results + assert "statistical_test" in results + assert "mse" in results["model_a_metrics"] + assert "r2" in results["model_a_metrics"] + assert "mse" in results["model_b_metrics"] + assert "r2" in results["model_b_metrics"] + +def test_report_generation(classification_data): + """Test report generation functionality.""" + X_train, X_test, y_train, y_test = classification_data + + # Create and train models + model_a = RandomForestClassifier(n_estimators=100, random_state=42) + model_b = RandomForestClassifier(n_estimators=50, random_state=42) + model_a.fit(X_train, y_train) + model_b.fit(X_train, y_train) + + # Generate report + comparison = ModelComparison(model_a, model_b, test_type="classification") + results = comparison.compare_predictions(X_test, y_test) + report = comparison.generate_report(results) + + # Check report content + assert isinstance(report, str) + assert "Model A/B Testing Results" in report + assert "Statistical Test Results" in report + assert "Model A Accuracy" in report + assert "Model B Accuracy" in report + assert "P-value" in report \ No newline at end of file diff --git a/igel/tests/test_igel/test_igel.py b/igel/tests/test_igel/test_igel.py new file mode 100644 index 0000000..d7031a8 --- /dev/null +++ b/igel/tests/test_igel/test_igel.py @@ -0,0 +1,41 @@ +#!/usr/bin/env python + +"""Tests for `igel` package.""" + +import os + +import pytest +from igel import Igel + +from .constants import Constants +from .helper import remove_folder +from .mock import MockCliArgs + +os.chdir(os.path.dirname(__file__)) + + +@pytest.fixture +def mock_args(): + yield MockCliArgs + remove_folder(Constants.model_results_dir) + assert Constants.model_results_dir.exists() == False + + +def test_fit(mock_args): + """ + test the fit model functionality + """ + assert mock_args is not None + Igel(**mock_args.fit) + assert Constants.model_results_dir.exists() == True + assert Constants.description_file.exists() == True + assert Constants.evaluation_file.exists() == False + +def test_export(mock_args): + """ + test the export model functionality + """ + assert mock_args is not None + Igel(**mock_args.fit) + Igel(**mock_args.export) + assert Constants.onnx_model_file.exists() == True diff --git a/igel/tox.ini b/igel/tox.ini new file mode 100644 index 0000000..934b615 --- /dev/null +++ b/igel/tox.ini @@ -0,0 +1,25 @@ +[tox] +envlist = py37, py38, flake8 + +[travis] +python = + 3.8: py38 + 3.7: py37 + +[testenv:flake8] +basepython = python +deps = flake8 +commands = flake8 igel tests + +[testenv] +setenv = + PYTHONPATH = {toxinidir} +deps = + -r{toxinidir}/requirements_dev.txt +; If you want to make tox run the tests with the same versions, create a +; requirements.txt with the pinned versions and uncomment the following line: +; -r{toxinidir}/requirements.txt +commands = + pip install -U pip + pytest --basetemp={envtmpdir} + diff --git a/infrastructure/cloudformation/README.md b/infrastructure/cloudformation/README.md new file mode 100644 index 0000000..68cae18 --- /dev/null +++ b/infrastructure/cloudformation/README.md @@ -0,0 +1,19 @@ +# CloudFormation ML Infrastructure + +This template provisions basic AWS infrastructure for ML workloads: +- S3 bucket for data/model storage +- EC2 instance for training/inference + +## Usage + +1. Go to the AWS CloudFormation console. +2. Upload `ml-infra.yaml` as a new stack. +3. Provide the required parameters (bucket name, AMI ID, instance type). +4. Launch the stack. + +## Cost Optimization +- Use spot instances or smaller instance types for cost savings. +- Set S3 lifecycle policies for data retention. + +## Monitoring +- Add CloudWatch alarms and logging as needed. \ No newline at end of file diff --git a/infrastructure/cloudformation/ml-infra.yaml b/infrastructure/cloudformation/ml-infra.yaml new file mode 100644 index 0000000..e6cd449 --- /dev/null +++ b/infrastructure/cloudformation/ml-infra.yaml @@ -0,0 +1,33 @@ +AWSTemplateFormatVersion: '2010-09-09' +Description: Basic ML infrastructure with S3 and EC2 +Parameters: + S3BucketName: + Type: String + Description: Name for the S3 bucket + AmiId: + Type: AWS::EC2::Image::Id + Description: AMI ID for EC2 instance + InstanceType: + Type: String + Default: t2.micro + Description: EC2 instance type +Resources: + MLDataBucket: + Type: AWS::S3::Bucket + Properties: + BucketName: !Ref S3BucketName + MLServer: + Type: AWS::EC2::Instance + Properties: + ImageId: !Ref AmiId + InstanceType: !Ref InstanceType + Tags: + - Key: Name + Value: ML-Server +Outputs: + S3BucketArn: + Value: !GetAtt MLDataBucket.Arn + Description: ARN of the S3 bucket + EC2InstanceId: + Value: !Ref MLServer + Description: ID of the EC2 instance \ No newline at end of file diff --git a/infrastructure/terraform/README.md b/infrastructure/terraform/README.md new file mode 100644 index 0000000..c5eedc2 --- /dev/null +++ b/infrastructure/terraform/README.md @@ -0,0 +1,24 @@ +# Terraform ML Infrastructure + +This template provisions basic AWS infrastructure for ML workloads: +- S3 bucket for data/model storage +- EC2 instance for training/inference + +## Usage + +1. Install [Terraform](https://www.terraform.io/). +2. Configure your AWS credentials. +3. Edit `variables.tf` to set your region, bucket name, AMI ID, and instance type. +4. Run: + ```sh + terraform init + terraform apply + ``` +5. Resources will be created in your AWS account. + +## Cost Optimization +- Use spot instances or smaller instance types for cost savings. +- Set S3 lifecycle policies for data retention. + +## Monitoring +- Add CloudWatch alarms and logging as needed. \ No newline at end of file diff --git a/infrastructure/terraform/main.tf b/infrastructure/terraform/main.tf new file mode 100644 index 0000000..1d05936 --- /dev/null +++ b/infrastructure/terraform/main.tf @@ -0,0 +1,18 @@ +provider "aws" { + region = var.region +} + +resource "aws_s3_bucket" "ml_data" { + bucket = var.s3_bucket_name + lifecycle { + prevent_destroy = false + } +} + +resource "aws_instance" "ml_server" { + ami = var.ami_id + instance_type = var.instance_type + tags = { + Name = "ML-Server" + } +} \ No newline at end of file diff --git a/infrastructure/terraform/outputs.tf b/infrastructure/terraform/outputs.tf new file mode 100644 index 0000000..d0311c4 --- /dev/null +++ b/infrastructure/terraform/outputs.tf @@ -0,0 +1,7 @@ +output "s3_bucket_arn" { + value = aws_s3_bucket.ml_data.arn +} + +output "ec2_instance_id" { + value = aws_instance.ml_server.id +} \ No newline at end of file diff --git a/infrastructure/terraform/variables.tf b/infrastructure/terraform/variables.tf new file mode 100644 index 0000000..c997dad --- /dev/null +++ b/infrastructure/terraform/variables.tf @@ -0,0 +1,19 @@ +variable "region" { + description = "AWS region" + type = string +} + +variable "s3_bucket_name" { + description = "Name for the S3 bucket" + type = string +} + +variable "ami_id" { + description = "AMI ID for EC2 instance" + type = string +} + +variable "instance_type" { + description = "EC2 instance type" + type = string +} \ No newline at end of file diff --git a/tests/test_feature_engineering.py b/tests/test_feature_engineering.py new file mode 100644 index 0000000..458497f Binary files /dev/null and b/tests/test_feature_engineering.py differ diff --git a/tests/test_few_shot_learning.py b/tests/test_few_shot_learning.py new file mode 100644 index 0000000..168c4c3 --- /dev/null +++ b/tests/test_few_shot_learning.py @@ -0,0 +1,463 @@ +""" +Tests for few-shot learning functionality in igel. +""" + +import pytest +import numpy as np +import pandas as pd +from sklearn.datasets import make_classification +from sklearn.model_selection import train_test_split +from sklearn.metrics import accuracy_score + +from igel.few_shot_learning import ( + MAMLClassifier, + PrototypicalNetwork, + DomainAdaptation, + TransferLearning, + create_few_shot_dataset, + evaluate_few_shot_model +) + + +class TestMAMLClassifier: + """Test cases for MAMLClassifier.""" + + def setup_method(self): + """Set up test data.""" + X, y = make_classification( + n_samples=200, + n_features=10, + n_classes=4, + n_clusters_per_class=1, + random_state=42 + ) + self.X = X + self.y = y + + def test_maml_initialization(self): + """Test MAML classifier initialization.""" + maml = MAMLClassifier( + inner_lr=0.01, + outer_lr=0.001, + num_tasks=5, + shots_per_task=3, + inner_steps=3, + meta_epochs=10 + ) + + assert maml.inner_lr == 0.01 + assert maml.outer_lr == 0.001 + assert maml.num_tasks == 5 + assert maml.shots_per_task == 3 + assert maml.inner_steps == 3 + assert maml.meta_epochs == 10 + assert not maml.is_fitted + + def test_maml_fit(self): + """Test MAML training.""" + maml = MAMLClassifier( + num_tasks=3, + shots_per_task=2, + inner_steps=2, + meta_epochs=5 # Small number for testing + ) + + maml.fit(self.X, self.y) + + assert maml.is_fitted + assert hasattr(maml, 'classes_') + assert hasattr(maml, 'n_classes_') + assert len(maml.classes_) == 4 + + def test_maml_predict(self): + """Test MAML prediction.""" + maml = MAMLClassifier( + num_tasks=3, + shots_per_task=2, + inner_steps=2, + meta_epochs=5 + ) + + maml.fit(self.X, self.y) + + # Test prediction + X_test = self.X[:10] + predictions = maml.predict(X_test) + + assert len(predictions) == 10 + assert all(pred in maml.classes_ for pred in predictions) + + def test_maml_predict_proba(self): + """Test MAML probability prediction.""" + maml = MAMLClassifier( + num_tasks=3, + shots_per_task=2, + inner_steps=2, + meta_epochs=5 + ) + + maml.fit(self.X, self.y) + + # Test probability prediction + X_test = self.X[:10] + probas = maml.predict_proba(X_test) + + assert probas.shape == (10, 4) + assert np.allclose(probas.sum(axis=1), 1.0, atol=1e-6) + + +class TestPrototypicalNetwork: + """Test cases for PrototypicalNetwork.""" + + def setup_method(self): + """Set up test data.""" + X, y = make_classification( + n_samples=200, + n_features=10, + n_classes=4, + n_clusters_per_class=1, + random_state=42 + ) + self.X = X + self.y = y + + def test_prototypical_network_initialization(self): + """Test Prototypical Network initialization.""" + proto_net = PrototypicalNetwork( + embedding_dim=32, + num_tasks=5, + shots_per_task=3, + meta_epochs=10 + ) + + assert proto_net.embedding_dim == 32 + assert proto_net.num_tasks == 5 + assert proto_net.shots_per_task == 3 + assert proto_net.meta_epochs == 10 + assert not proto_net.is_fitted + + def test_prototypical_network_fit(self): + """Test Prototypical Network training.""" + proto_net = PrototypicalNetwork( + embedding_dim=16, + num_tasks=3, + shots_per_task=2, + meta_epochs=5 # Small number for testing + ) + + proto_net.fit(self.X, self.y) + + assert proto_net.is_fitted + assert hasattr(proto_net, 'classes_') + assert hasattr(proto_net, 'n_classes_') + assert len(proto_net.classes_) == 4 + + def test_prototypical_network_predict(self): + """Test Prototypical Network prediction.""" + proto_net = PrototypicalNetwork( + embedding_dim=16, + num_tasks=3, + shots_per_task=2, + meta_epochs=5 + ) + + proto_net.fit(self.X, self.y) + + # Test prediction + X_test = self.X[:10] + predictions = proto_net.predict(X_test) + + assert len(predictions) == 10 + + +class TestDomainAdaptation: + """Test cases for DomainAdaptation.""" + + def setup_method(self): + """Set up test data.""" + # Create source domain data + X_source, y_source = make_classification( + n_samples=100, + n_features=10, + n_classes=3, + random_state=42 + ) + + # Create target domain data with some shift + X_target, y_target = make_classification( + n_samples=50, + n_features=10, + n_classes=3, + random_state=123 + ) + + self.X_source = X_source + self.y_source = y_source + self.X_target = X_target + self.y_target = y_target + + def test_domain_adaptation_initialization(self): + """Test DomainAdaptation initialization.""" + from sklearn.ensemble import RandomForestClassifier + + base_model = RandomForestClassifier(n_estimators=10, random_state=42) + adapter = DomainAdaptation(base_model) + + assert adapter.base_model == base_model + assert not adapter.is_adapted + + def test_fine_tune_adaptation(self): + """Test fine-tuning domain adaptation.""" + from sklearn.ensemble import RandomForestClassifier + + base_model = RandomForestClassifier(n_estimators=10, random_state=42) + adapter = DomainAdaptation(base_model) + + # Train base model on source + base_model.fit(self.X_source, self.y_source) + + # Perform adaptation + adapted_model = adapter.adapt_model( + self.X_source, self.y_source, + self.X_target, self.y_target, + adaptation_method='fine_tune' + ) + + assert adapter.is_adapted + assert adapted_model is not None + + def test_maml_adaptation(self): + """Test MAML-based domain adaptation.""" + from sklearn.ensemble import RandomForestClassifier + + base_model = RandomForestClassifier(n_estimators=10, random_state=42) + adapter = DomainAdaptation(base_model) + + # Perform MAML adaptation + adapted_model = adapter.adapt_model( + self.X_source, self.y_source, + self.X_target, self.y_target, + adaptation_method='maml' + ) + + assert adapter.is_adapted + assert adapted_model is not None + + +class TestTransferLearning: + """Test cases for TransferLearning.""" + + def setup_method(self): + """Set up test data.""" + # Create source data + X_source, y_source = make_classification( + n_samples=100, + n_features=10, + n_classes=4, + random_state=42 + ) + + # Create target data + X_target, y_target = make_classification( + n_samples=50, + n_features=10, + n_classes=3, + random_state=123 + ) + + self.X_source = X_source + self.y_source = y_source + self.X_target = X_target + self.y_target = y_target + + def test_transfer_learning_initialization(self): + """Test TransferLearning initialization.""" + from sklearn.ensemble import RandomForestClassifier + + source_model = RandomForestClassifier(n_estimators=10, random_state=42) + transfer = TransferLearning(source_model) + + assert transfer.base_model == source_model + + def test_feature_extraction_transfer(self): + """Test feature extraction transfer learning.""" + from sklearn.ensemble import RandomForestClassifier + + # Train source model + source_model = RandomForestClassifier(n_estimators=10, random_state=42) + source_model.fit(self.X_source, self.y_source) + + transfer = TransferLearning(source_model) + + # Perform feature extraction transfer + transfer_model = transfer.create_transfer_model( + source_model, self.X_target, self.y_target, + method='feature_extraction' + ) + + assert transfer_model is not None + + # Test prediction + predictions = transfer_model.predict(self.X_target[:10]) + assert len(predictions) == 10 + + def test_fine_tuning_transfer(self): + """Test fine-tuning transfer learning.""" + from sklearn.ensemble import RandomForestClassifier + + # Train source model + source_model = RandomForestClassifier(n_estimators=10, random_state=42) + source_model.fit(self.X_source, self.y_source) + + transfer = TransferLearning(source_model) + + # Perform fine-tuning transfer + transfer_model = transfer.create_transfer_model( + source_model, self.X_target, self.y_target, + method='fine_tuning' + ) + + assert transfer_model is not None + + +class TestUtilityFunctions: + """Test cases for utility functions.""" + + def setup_method(self): + """Set up test data.""" + X, y = make_classification( + n_samples=200, + n_features=10, + n_classes=6, + random_state=42 + ) + self.X = X + self.y = y + + def test_create_few_shot_dataset(self): + """Test few-shot dataset creation.""" + tasks = create_few_shot_dataset( + self.X, self.y, + n_way=3, + k_shot=5, + n_query=5 + ) + + assert len(tasks) > 0 + + # Check task structure + support_X, support_y, query_X, query_y = tasks[0] + + assert support_X.shape[0] == 15 # 3 classes * 5 shots + assert query_X.shape[0] == 15 # 3 classes * 5 queries + assert len(support_y) == 15 + assert len(query_y) == 15 + + # Check that classes are relabeled as 0, 1, 2 + assert set(support_y) == {0, 1, 2} + assert set(query_y) == {0, 1, 2} + + def test_evaluate_few_shot_model(self): + """Test few-shot model evaluation.""" + # Create a simple mock model + class MockModel: + def __init__(self): + self.is_fitted = True + + def predict(self, X): + return np.random.randint(0, 3, len(X)) + + model = MockModel() + + # Create tasks + tasks = create_few_shot_dataset( + self.X, self.y, + n_way=2, + k_shot=3, + n_query=3 + ) + + # Evaluate model + results = evaluate_few_shot_model(model, tasks) + + assert 'mean_accuracy' in results + assert 'std_accuracy' in results + assert 'accuracies' in results + assert 0 <= results['mean_accuracy'] <= 1 + assert len(results['accuracies']) == len(tasks) + + +class TestIntegration: + """Integration tests for few-shot learning.""" + + def test_end_to_end_maml(self): + """Test end-to-end MAML workflow.""" + # Create data + X, y = make_classification( + n_samples=300, + n_features=15, + n_classes=5, + random_state=42 + ) + + # Split data + X_train, X_test, y_train, y_test = train_test_split( + X, y, test_size=0.2, random_state=42 + ) + + # Train MAML + maml = MAMLClassifier( + num_tasks=5, + shots_per_task=3, + inner_steps=3, + meta_epochs=10 + ) + + maml.fit(X_train, y_train) + + # Create evaluation tasks + tasks = create_few_shot_dataset(X_test, y_test, n_way=2, k_shot=3, n_query=3) + + # Evaluate + results = evaluate_few_shot_model(maml, tasks) + + assert results['mean_accuracy'] >= 0 + assert results['std_accuracy'] >= 0 + + def test_end_to_end_prototypical_network(self): + """Test end-to-end Prototypical Network workflow.""" + # Create data + X, y = make_classification( + n_samples=300, + n_features=15, + n_classes=5, + random_state=42 + ) + + # Split data + X_train, X_test, y_train, y_test = train_test_split( + X, y, test_size=0.2, random_state=42 + ) + + # Train Prototypical Network + proto_net = PrototypicalNetwork( + embedding_dim=32, + num_tasks=5, + shots_per_task=3, + meta_epochs=10 + ) + + proto_net.fit(X_train, y_train) + + # Create evaluation tasks + tasks = create_few_shot_dataset(X_test, y_test, n_way=2, k_shot=3, n_query=3) + + # Evaluate + results = evaluate_few_shot_model(proto_net, tasks) + + assert results['mean_accuracy'] >= 0 + assert results['std_accuracy'] >= 0 + + +if __name__ == "__main__": + pytest.main([__file__]) \ No newline at end of file