Feature/enhanced ab testing compatibility #353
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 14 commits
June 22, 2025 00:40
- Implement Model-Agnostic Meta-Learning (MAML) classifier - Add Prototypical Networks for few-shot learning - Create domain adaptation utilities with fine-tuning and MAML methods - Add transfer learning capabilities with feature extraction and fine-tuning - Include utility functions for creating and evaluating few-shot tasks - Add CLI commands: few-shot-learn, domain-adapt, transfer-learn - Update models_dict to include few-shot learning algorithms - Add few_shot_learning as supported model type - Create comprehensive documentation and examples - Add complete test suite for all few-shot learning components - Update README with new features and model table This addresses GitHub issue nidhaloff#237 'Add Support for Few-Shot Learning'
…em (Issue nidhaloff#233) - Implement MLflow-like experiment tracking with ExperimentTracker class - Add model versioning with lineage tracking and metadata management - Create experiment visualization and analysis capabilities - Include SQLite database for experiment and model metadata storage - Add support for metric tracking, parameter logging, and model logging - Implement experiment comparison and visualization tools - Add model lineage visualization and deployment tracking - Include interactive Plotly dashboards for experiment analysis - Support for experiment export and model version management - Add comprehensive documentation and examples This addresses GitHub issue nidhaloff#233 'Create Model Versioning and Experiment Tracking'
- Add SyntheticDataGenerator class for creating test datasets - Support for classification and regression data generation - Quick function for generating sample datasets - Addresses GitHub issue nidhaloff#285 - Add Support for Synthetic Data Generation
- Implemented AutoRetrainer class with performance-based and time-based strategies - Added RetrainingScheduler for scheduling retraining jobs - Created example configuration and demo files - Added test structure - Addresses GitHub Issue nidhaloff#339
- Enhanced A/B testing with comprehensive statistical tests, confidence intervals, and visualizations - Added backward compatibility layer with CompatibilityManager - Improved CLI with new options for visualization, export, and legacy mode - Added support for multiple statistical tests (McNemar, Chi-square, Wilcoxon, paired t-test) - Enhanced reporting with detailed metrics and recommendations - Added visualization capabilities for model comparison results - Implemented robust import fallback system for different igel versions Resolves issues nidhaloff#330 and nidhaloff#331
- Created comprehensive ensemble framework with voting, stacking, blending, bagging, and boosting - Added automatic ensemble selection based on data characteristics - Implemented model compression with pruning, quantization, knowledge distillation, and feature selection - Added model optimization for accuracy, speed, memory, and balanced performance - Enhanced CLI with create-ensemble, predict-ensemble, compress-model, and optimize-model commands - Added comprehensive reporting and model persistence capabilities - Implemented performance comparison and compression statistics Resolves issues nidhaloff#332 and nidhaloff#333
- Created comprehensive model explainability framework with multiple explanation methods - Added support for feature importance, partial dependence, SHAP values, LIME explanations, and permutation importance - Implemented static and interactive dashboards for model interpretation - Added visualizations for feature importance, partial dependence plots, correlation matrices, and model performance - Enhanced CLI with explain-model command supporting multiple explanation types - Added interactive dashboard with Dash and Plotly for real-time model exploration - Implemented comprehensive reporting and explanation persistence - Added support for both static and interactive dashboard generation Resolves issue nidhaloff#334
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enhanced A/B Testing Framework & Backward Compatibility
This PR implements significant enhancements to igel's A/B testing capabilities while ensuring full backward compatibility.
Key Features Added
Enhanced A/B Testing Framework
Backward Compatibility Layer
📝 Resolves