Config v6 (#50)

* Expose Python 3.11 support

* add a tox.ini config for easier local testing

* Drop unsupported Python versions and update syntax accordingly

* Add Flake8 linting to tox

* Fix semver deprecations warnings

* Export typing using PEP-561 py.typed

* Add typing check and expose linting settings

* Simplify github actions workflow

- run test and coverage in a single pass
- use official codecov action to upload coverage
- add typing analysis
- run flake8 once and rely on settings

* Fix all lints

* Update contributing guide and document `tox` usage

* Ignore python:S4790 intentional Sonar errors

* reverse commit: Drop unsupported Python versions and update syntax accordingly

* update CONTRIBUTING.md

* remove MyPy check

* universal bdist_wheel

* intro config v6 json format

* add comparators

* conditions check

* dependent flag logging into the same log_entries

* configclient get_config fixes

* dependency loop check

* testmatrix comparators_v6

* testmatrix segments

* testmatrix dependent flag

* testmatrix: AndOr

* dependent flag logging

* comments

* TODO: percentage_rule_attribute

* sdk key validation check

* percentage_rule_attribute log

* move sha256 calculation into a function

* finalize no percentage_rule_attribute error handling

* introduce typed value in override + test fixes

* linter fixes

* cleanup

* github test fix

* custom percentage attribute

* IS NOT IN SEGMENT fix

* operator updates

* update tests

* lint fixes

* circular dependency test

* new evaluation logging (WIP)

* github action: python 2.7 support

* fix user json key order on python 2.7

* Remove the u prefix from unicode strings on python 2.7 in the eval log tests

* evaluation logging

* lint fixes

* fix tests

* test_options_within_targeting_rule

* lint fix

* typo fix

* handling the modified config json format

* evaluation log test + generator + data

* lint fix

* Adjust evaluation and update evaluation tests

* In case of local only flag overrides mode, we accept any SDK Key format

* rename comparators

* NOT STARTS WITH ANY OF (hashed), NOT ENDS WITH ANY OF (hashed) comparators

* eval log tests: validation error handling

* lint fixes

* consistent trim logic during evaluation

* log fix

* test incorrect json

* evaluation log update: hashed value + max 10 length lists

* indentation fix

* matched_evaluation_rule -> matched_targeting_rule, matched_evaluation_percentage_rule -> matched_percentage_rule

* evallogging: list logging fix

* update matrix tests + eval log tests

* Fix list_truncation.txt

* Turn off python 2.7 build

* attr_value_from_datetime + attr_value_from_list

* inline salt, segment for handling flag override

* fix python 3.5 test

* check if the prerequisite key exists

* remove unnecessary served_value get

* add `any of` tests to testmatrix_comparators_v6

* rename config members (comparision_rule -> user_condition, segment_rule -> segment_condition)

* config descriptor

* github-ci fix

* fix python 2.7 tests

* github ci: python 3.5

* Adjust config model and tests to config v6 schema changes

* matrix test update + new cleartext comparators

* unicode tests

* segments_old matrix test + segment eval log fix

* python 2.7 unicode support

* fix evaluation log test on python <= 3.5

* type mismatched user attribute warning

* python 3.12 support

* lint fix + review fixes

* python 2.7 fix

* github ci: win test fix

* review fixes + exceptions in prerequisite flag evaluation

* lint fix

* don't force users to pass user object attributes as strings

* test: evaluation_details_matched_evaluation_rule_and_percentage_option

* DefaultValue and SettingType mismatch warning

* forced setting_type check

* python 2.7 support (timezone, unicode/str handling)

* Read cache upon each setting read (lazy cache fix)

* fix coding convention

* bump version 9.0.0

* ConfigService's refresh offline warning

* comment updates

* remove unsupported value error log

---------

Co-authored-by: Axel H <[email protected]>

Author: kp-cat

.github/workflows/python-ci.yml

@@ -14,10 +14,11 @@ on:
 jobs:
   test:
 
-    runs-on: ubuntu-20.04
+    runs-on: ${{ matrix.os }}
     strategy:
       matrix:
-        python-version: ["2.7", "3.5", "3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]
+        python-version: ["2.7", "3.5", "3.6", "3.7", "3.8", "3.9", "3.10", "3.11", "3.12"]
+        os: [ windows-latest, macos-latest, ubuntu-20.04 ]
 
     steps:
     - uses: actions/checkout@v3
@@ -36,7 +37,7 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip 
-        pip install pytest pytest-cov mock flake8
+        pip install pytest pytest-cov parameterized mock flake8
         pip install -r requirements.txt
 
     - name: Lint with flake8

configcatclient/config.py

@@ -0,0 +1,293 @@
+from enum import IntEnum
+
+CONFIG_FILE_NAME = 'config_v6'
+SERIALIZATION_FORMAT_VERSION = 'v2'
+
+# Config
+PREFERENCES = 'p'
+SEGMENTS = 's'
+FEATURE_FLAGS = 'f'
+
+# Preferences
+BASE_URL = 'u'
+REDIRECT = 'r'
+SALT = 's'
+
+# Segment
+SEGMENT_NAME = 'n'  # The first 4 characters of the Segment's name
+SEGMENT_CONDITIONS = 'r'  # The list of segment rule conditions (has a logical AND relation between the items).
+
+# Segment Condition (User Condition)
+COMPARISON_ATTRIBUTE = 'a'  # The attribute of the user object that should be used to evaluate this rule
+COMPARATOR = 'c'
+
+# Feature flag (Evaluation Formula)
+SETTING_TYPE = 't'  # 0 = bool, 1 = string, 2 = int, 3 = double
+PERCENTAGE_RULE_ATTRIBUTE = 'a'  # Percentage rule evaluation hashes this attribute of the User object to calculate the buckets
+TARGETING_RULES = 'r'  # Targeting Rules (Logically connected by OR)
+PERCENTAGE_OPTIONS = 'p'  # Percentage Options without conditions
+VALUE = 'v'
+VARIATION_ID = 'i'
+INLINE_SALT = 'inline_salt'
+
+# Targeting Rule (Evaluation Rule)
+CONDITIONS = 'c'
+SERVED_VALUE = 's'  # Value and Variation ID
+TARGETING_RULE_PERCENTAGE_OPTIONS = 'p'
+
+# Condition
+USER_CONDITION = 'u'
+SEGMENT_CONDITION = 's'  # Segment targeting rule
+PREREQUISITE_FLAG_CONDITION = 'p'  # Prerequisite flag targeting rule
+
+# Segment Condition
+SEGMENT_INDEX = 's'
+SEGMENT_COMPARATOR = 'c'
+INLINE_SEGMENT = 'inline_segment'
+
+# Prerequisite Flag Condition
+PREREQUISITE_FLAG_KEY = 'f'
+PREREQUISITE_COMPARATOR = 'c'
+
+# Percentage Option
+PERCENTAGE = 'p'
+
+# Value
+BOOL_VALUE = 'b'
+STRING_VALUE = 's'
+INT_VALUE = 'i'
+DOUBLE_VALUE = 'd'
+STRING_LIST_VALUE = 'l'
+UNSUPPORTED_VALUE = 'unsupported_value'
+
+
+def get_value(dictionary, setting_type):
+    value_descriptor = dictionary.get(VALUE)
+    if value_descriptor is None:
+        raise ValueError('Value is missing')
+
+    if setting_type not in list(map(int, SettingType)):
+        raise ValueError('Unsupported setting type')
+
+    expected_value_type, expected_py_type = SettingType.get_type_info(setting_type)
+    if expected_value_type is None:
+        raise ValueError('Unsupported setting type')
+
+    value = value_descriptor.get(expected_value_type)
+    if value is None:
+        raise ValueError('Setting value is not of the expected type %s' % expected_py_type)
+
+    return value
+
+
+def get_value_type(dictionary):
+    value = dictionary.get(VALUE)
+    if value is not None:
+        if value.get(BOOL_VALUE) is not None:
+            return bool
+        if value.get(STRING_VALUE) is not None:
+            return str
+        if value.get(INT_VALUE) is not None:
+            return int
+        if value.get(DOUBLE_VALUE) is not None:
+            return float
+
+    return None
+
+
+class SettingType(IntEnum):
+    BOOL = 0
+    STRING = 1
+    INT = 2
+    DOUBLE = 3
+
+    @staticmethod
+    def get_type_info(setting_type):
+        return setting_type_mapping.get(setting_type, (None, None))
+
+    @staticmethod
+    def from_type(object_type):
+        if object_type is bool:
+            return SettingType.BOOL
+        if object_type is str:
+            return SettingType.STRING
+        if object_type is int:
+            return SettingType.INT
+        if object_type is float:
+            return SettingType.DOUBLE
+
+        return None
+
+    @staticmethod
+    def to_type(setting_type):
+        return SettingType.get_type_info(setting_type)[1]
+
+    @staticmethod
+    def to_value_type(setting_type):
+        return SettingType.get_type_info(setting_type)[0]
+
+
+setting_type_mapping = {
+    SettingType.BOOL: (BOOL_VALUE, bool),
+    SettingType.STRING: (STRING_VALUE, str),
+    SettingType.INT: (INT_VALUE, int),
+    SettingType.DOUBLE: (DOUBLE_VALUE, float)
+}
+
+
+class PrerequisiteComparator(IntEnum):
+    EQUALS = 0
+    NOT_EQUALS = 1
+
+
+class SegmentComparator(IntEnum):
+    IS_IN = 0
+    IS_NOT_IN = 1
+
+
+class Comparator(IntEnum):
+    IS_ONE_OF = 0
+    IS_NOT_ONE_OF = 1
+    CONTAINS_ANY_OF = 2
+    NOT_CONTAINS_ANY_OF = 3
+    IS_ONE_OF_SEMVER = 4
+    IS_NOT_ONE_OF_SEMVER = 5
+    LESS_THAN_SEMVER = 6
+    LESS_THAN_OR_EQUAL_SEMVER = 7
+    GREATER_THAN_SEMVER = 8
+    GREATER_THAN_OR_EQUAL_SEMVER = 9
+    EQUALS_NUMBER = 10
+    NOT_EQUALS_NUMBER = 11
+    LESS_THAN_NUMBER = 12
+    LESS_THAN_OR_EQUAL_NUMBER = 13
+    GREATER_THAN_NUMBER = 14
+    GREATER_THAN_OR_EQUAL_NUMBER = 15
+    IS_ONE_OF_HASHED = 16
+    IS_NOT_ONE_OF_HASHED = 17
+    BEFORE_DATETIME = 18
+    AFTER_DATETIME = 19
+    EQUALS_HASHED = 20
+    NOT_EQUALS_HASHED = 21
+    STARTS_WITH_ANY_OF_HASHED = 22
+    NOT_STARTS_WITH_ANY_OF_HASHED = 23
+    ENDS_WITH_ANY_OF_HASHED = 24
+    NOT_ENDS_WITH_ANY_OF_HASHED = 25
+    ARRAY_CONTAINS_ANY_OF_HASHED = 26
+    ARRAY_NOT_CONTAINS_ANY_OF_HASHED = 27
+    EQUALS = 28
+    NOT_EQUALS = 29
+    STARTS_WITH_ANY_OF = 30
+    NOT_STARTS_WITH_ANY_OF = 31
+    ENDS_WITH_ANY_OF = 32
+    NOT_ENDS_WITH_ANY_OF = 33
+    ARRAY_CONTAINS_ANY_OF = 34
+    ARRAY_NOT_CONTAINS_ANY_OF = 35
+
+
+COMPARATOR_TEXTS = [
+    'IS ONE OF',                  # IS_ONE_OF
+    'IS NOT ONE OF',              # IS_NOT_ONE_OF
+    'CONTAINS ANY OF',            # CONTAINS_ANY_OF
+    'NOT CONTAINS ANY OF',        # NOT_CONTAINS_ANY_OF
+    'IS ONE OF',                  # IS_ONE_OF_SEMVER
+    'IS NOT ONE OF',              # IS_NOT_ONE_OF_SEMVER
+    '<',                          # LESS_THAN_SEMVER
+    '<=',                         # LESS_THAN_OR_EQUAL_SEMVER
+    '>',                          # GREATER_THAN_SEMVER
+    '>=',                         # GREATER_THAN_OR_EQUAL_SEMVER
+    '=',                          # EQUALS_NUMBER
+    '!=',                         # NOT_EQUALS_NUMBER
+    '<',                          # LESS_THAN_NUMBER
+    '<=',                         # LESS_THAN_OR_EQUAL_NUMBER
+    '>',                          # GREATER_THAN_NUMBER
+    '>=',                         # GREATER_THAN_OR_EQUAL_NUMBER
+    'IS ONE OF',                  # IS_ONE_OF_HASHED
+    'IS NOT ONE OF',              # IS_NOT_ONE_OF_HASHED
+    'BEFORE',                     # BEFORE_DATETIME
+    'AFTER',                      # AFTER_DATETIME
+    'EQUALS',                     # EQUALS_HASHED
+    'NOT EQUALS',                 # NOT_EQUALS_HASHED
+    'STARTS WITH ANY OF',         # STARTS_WITH_ANY_OF_HASHED
+    'NOT STARTS WITH ANY OF',     # NOT_STARTS_WITH_ANY_OF_HASHED
+    'ENDS WITH ANY OF',           # ENDS_WITH_ANY_OF_HASHED
+    'NOT ENDS WITH ANY OF',       # NOT_ENDS_WITH_ANY_OF_HASHED
+    'ARRAY CONTAINS ANY OF',      # ARRAY_CONTAINS_ANY_OF_HASHED
+    'ARRAY NOT CONTAINS ANY OF',  # ARRAY_NOT_CONTAINS_ANY_OF_HASHED
+    'EQUALS',                     # EQUALS
+    'NOT EQUALS',                 # NOT_EQUALS
+    'STARTS WITH ANY OF',         # STARTS_WITH_ANY_OF
+    'NOT STARTS WITH ANY OF',     # NOT_STARTS_WITH_ANY_OF
+    'ENDS WITH ANY OF',           # ENDS_WITH_ANY_OF
+    'NOT ENDS WITH ANY OF',       # NOT_ENDS_WITH_ANY_OF
+    'ARRAY CONTAINS ANY OF',      # ARRAY_CONTAINS_ANY_OF
+    'ARRAY NOT CONTAINS ANY OF'   # ARRAY_NOT_CONTAINS_ANY_OF
+]
+COMPARISON_VALUES = [
+    STRING_LIST_VALUE,  # IS_ONE_OF
+    STRING_LIST_VALUE,  # IS_NOT_ONE_OF
+    STRING_LIST_VALUE,  # CONTAINS_ANY_OF
+    STRING_LIST_VALUE,  # NOT_CONTAINS_ANY_OF
+    STRING_LIST_VALUE,  # IS_ONE_OF_SEMVER
+    STRING_LIST_VALUE,  # IS_NOT_ONE_OF_SEMVER
+    STRING_VALUE,       # LESS_THAN_SEMVER
+    STRING_VALUE,       # LESS_THAN_OR_EQUAL_SEMVER
+    STRING_VALUE,       # GREATER_THAN_SEMVER
+    STRING_VALUE,       # GREATER_THAN_OR_EQUAL_SEMVER
+    DOUBLE_VALUE,       # EQUALS_NUMBER
+    DOUBLE_VALUE,       # NOT_EQUALS_NUMBER
+    DOUBLE_VALUE,       # LESS_THAN_NUMBER
+    DOUBLE_VALUE,       # LESS_THAN_OR_EQUAL_NUMBER
+    DOUBLE_VALUE,       # GREATER_THAN_NUMBER
+    DOUBLE_VALUE,       # GREATER_THAN_OR_EQUAL_NUMBER
+    STRING_LIST_VALUE,  # IS_ONE_OF_HASHED
+    STRING_LIST_VALUE,  # IS_NOT_ONE_OF_HASHED
+    DOUBLE_VALUE,       # BEFORE_DATETIME
+    DOUBLE_VALUE,       # AFTER_DATETIME
+    STRING_VALUE,       # EQUALS_HASHED
+    STRING_VALUE,       # NOT_EQUALS_HASHED
+    STRING_LIST_VALUE,  # STARTS_WITH_ANY_OF_HASHED
+    STRING_LIST_VALUE,  # NOT_STARTS_WITH_ANY_OF_HASHED
+    STRING_LIST_VALUE,  # ENDS_WITH_ANY_OF_HASHED
+    STRING_LIST_VALUE,  # NOT_ENDS_WITH_ANY_OF_HASHED
+    STRING_LIST_VALUE,  # ARRAY_CONTAINS_ANY_OF_HASHED
+    STRING_LIST_VALUE,  # ARRAY_NOT_CONTAINS_ANY_OF_HASHED
+    STRING_VALUE,       # EQUALS
+    STRING_VALUE,       # NOT_EQUALS
+    STRING_LIST_VALUE,  # STARTS_WITH_ANY_OF
+    STRING_LIST_VALUE,  # NOT_STARTS_WITH_ANY_OF
+    STRING_LIST_VALUE,  # ENDS_WITH_ANY_OF
+    STRING_LIST_VALUE,  # NOT_ENDS_WITH_ANY_OF
+    STRING_LIST_VALUE,  # ARRAY_CONTAINS_ANY_OF
+    STRING_LIST_VALUE   # ARRAY_NOT_CONTAINS_ANY_OF
+]
+SEGMENT_COMPARATOR_TEXTS = ['IS IN SEGMENT', 'IS NOT IN SEGMENT']
+PREREQUISITE_COMPARATOR_TEXTS = ['EQUALS', 'DOES NOT EQUAL']
+
+
+def extend_config_with_inline_salt_and_segment(config):
+    """
+    Adds the inline salt and segment to the config.
+    When using flag overrides, the original salt and segment indexes may become invalid. Therefore, we copy the
+    object references to the locations where they are referenced and use these references instead of the indexes.
+    """
+    salt = config.get(PREFERENCES, {}).get(SALT, '')
+    segments = config.get(SEGMENTS, [])
+    settings = config.get(FEATURE_FLAGS, {})
+    for setting in settings.values():
+        if not isinstance(setting, dict):
+            continue
+
+        # add salt
+        setting[INLINE_SALT] = salt
+
+        # add segment to the segment conditions
+        targeting_rules = setting.get(TARGETING_RULES, [])
+        for targeting_rule in targeting_rules:
+            conditions = targeting_rule.get(CONDITIONS, [])
+            for condition in conditions:
+
+                segment_condition = condition.get(SEGMENT_CONDITION)
+                if segment_condition:
+                    segment_index = segment_condition.get(SEGMENT_INDEX)
+                    segment = segments[segment_index]
+                    segment_condition[INLINE_SEGMENT] = segment