Skip to content
Closed

V1 #75

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
326 commits
Select commit Hold shift + click to select a range
9f062d0
Merge pull request #82 from microavia/v1-js-implement
Roman- Apr 17, 2024
592c57d
fix empty structs for cpp<20
Apr 17, 2024
a84a182
Merge pull request #86 from microavia/v1_fix_cpp17_empty
Roman- Apr 17, 2024
7b7fa42
add c++ test executables for both c++20 and c++<20 standards
Apr 17, 2024
21cba68
Refactor messgen.ts to use Messgen class and enhance type handling, a…
meded7 Apr 17, 2024
3caaa8d
add error handling, invalid protocols and tests
Apr 17, 2024
b9d8e07
restructure validation tests
Apr 17, 2024
f99fc68
add cyclic reference test
Apr 17, 2024
493feb6
Merge pull request #88 from microavia/v1_improvements_and_tests
Roman- Apr 17, 2024
7e885ac
add validation as a separate js file
Apr 18, 2024
d774899
Merge remote-tracking branch 'origin/v1-variant' into v1_improvements
Apr 18, 2024
ba2d779
fixes for variant type class
Apr 18, 2024
e1cc225
remove unused code from cpp tests
Apr 18, 2024
500bc34
v1 variant implementation
Apr 18, 2024
71c3161
add python impl for VariantType
Apr 18, 2024
3624a2e
Add std::variant implementation
Roman- Apr 18, 2024
e4d2047
Add StructConverter and PrimitiveConverter for serialization and dese…
meded7 Apr 19, 2024
ebcf264
Implement NestedConverter with support for nested arrays and maps, en…
meded7 Apr 22, 2024
35a18bd
Implement map serialization/deserialization in NestedConverter, fix e…
meded7 Apr 22, 2024
d080b7a
Refactor converter structure and update tests for consistency across …
meded7 Apr 23, 2024
26bddcd
Implement EnumConverter and integration tests, refactor type handling…
meded7 Apr 23, 2024
765ce43
Optimize StructConverter by moving converter lookup to constructor, a…
meded7 Apr 24, 2024
aec8c30
remove std::variant support
Apr 24, 2024
c34e4c6
Merge pull request #90 from microavia/remove_variant_support
Roman- Apr 24, 2024
2ce35fa
add extra warnings (padding, array size)
Apr 24, 2024
05c0db5
Merge pull request #91 from microavia/extra_warnings
Roman- Apr 24, 2024
e4ddbe5
Extend serialization support for `bytes` type, refactor `EnumConverte…
meded7 Apr 24, 2024
36107b2
Refactor NestedConverter to use IValue type
meded7 Apr 24, 2024
47f2cf4
refactor test descriptions
meded7 Apr 24, 2024
3faca87
Add "proto_name" field to ProtocolJSON interface and JSON generator
meded7 Apr 25, 2024
d6447cf
Add serialization test data and improve testing scripts
meded7 Apr 25, 2024
4d56938
Update import paths and add new benchmark tests
meded7 Apr 26, 2024
dbf125b
Add default value methods to converters and initialize StructConverte…
meded7 Apr 26, 2024
df6890d
Remove benchmarking and messgen files in 'deserialize-variant' directory
meded7 Apr 26, 2024
953ae92
Add support for typed arrays in converters and tests
meded7 Apr 26, 2024
0c87d43
Refactor Messgen to support multiple protocols, improve serialization…
meded7 Apr 29, 2024
d8cd1d8
Extend Messgen with custom header serialization/deserialization suppo…
meded7 Apr 29, 2024
1e2097d
Remove unused Buffer import in Messgen.test.ts
meded7 Apr 29, 2024
76ec081
C++ dynamic bugfixes
DrTon Jul 1, 2024
853c3f1
Do not depend on non existent file
lachem Jul 29, 2024
dee728b
Tolerate trailing slashes in outdir
lachem Jul 29, 2024
f950f7d
Merge pull request #95 from microavia/v1-dispatch-fix-cmake-integration
DrTon Jul 31, 2024
dead7ed
Relax requirement check in dispatch
Aug 6, 2024
4b04d7b
Update cpp_generator.py
lachem Aug 6, 2024
ade89d2
Add code.append("")
lachem Aug 6, 2024
4b133d9
Merge pull request #97 from microavia/v1-dispatch-relax
DrTon Aug 6, 2024
ebac694
README updated
Oct 30, 2024
be052a7
Merge branch 'v1' into v1-dispatch
Oct 30, 2024
806b4c9
feat: typescript generator poc
Nov 11, 2024
fe977b2
feat: generate right imports between protocols
Nov 11, 2024
87745ce
refactor: fix review formatting comments
Nov 12, 2024
9c69fb9
Merge pull request #96 from microavia/v1-dispatch
DrTon Nov 12, 2024
2b53435
feat: add types map generator
Nov 12, 2024
fe920ba
refactor: split messgen js class to protocol manager and base messgen…
Nov 12, 2024
9a7c838
feat: add ability to pass types for js scheme
Nov 12, 2024
7017635
test: reformat test for struct converter
Nov 13, 2024
8018e02
test: reformat tests a bit
Nov 13, 2024
65533c9
refactor: fix protocl id extraction
Nov 13, 2024
06e3c3a
feat: support typed arrays and maps for ts generator
Nov 13, 2024
206962f
refactor: support empty structs for ts generator
Nov 14, 2024
842808e
test: basic converter modular tests
Nov 14, 2024
d84c808
Merge pull request #104 from microavia/v1-ts-generator
meded90 Nov 14, 2024
6cfbf28
Merge pull request #87 from microavia/v1-js-implement
meded90 Nov 14, 2024
bff6963
Add reflection helpers to messgen.h
Nov 12, 2024
05a0825
Generate members of and reflect_message
Nov 12, 2024
7f4769a
Enable github workflows for v1 branch
Nov 12, 2024
c2040c2
Install cmake and libgtest-dev
Nov 12, 2024
edca9d6
Use cmake for buil and run tests
Nov 12, 2024
7aced30
Add ninja & gcc deps to cpp workflow
Nov 12, 2024
de123e0
Rename Install gtest to Install dependencies
Nov 12, 2024
fd57394
Add build step
Nov 12, 2024
0fb852c
Add gtest_discover_tests
Nov 12, 2024
f2f187c
Fix build
Nov 12, 2024
86e1644
Improve test integration in CMake
Nov 12, 2024
89969f1
Add more cmake dirs to .gitignore
Nov 12, 2024
e2ddd4e
Integrate ctest into github workflow
Nov 12, 2024
47c1384
Fix zero size array serialization
Nov 12, 2024
719fc74
Fix some mypy findings
Nov 12, 2024
b502b3f
Include tuple, remove spare print
Nov 13, 2024
e81b131
Try using cache-apt helper to speed up install
Nov 13, 2024
b4f0f2f
Fix github workflow syntax
Nov 13, 2024
07ee76c
Test trigger to see cache speed up
Nov 13, 2024
fd98bb4
Print output of failed tests
Nov 13, 2024
7b7b462
Add support for enum reflection
Nov 13, 2024
dfaf9d5
Add unit tests for message reflection
Nov 13, 2024
3675cde
Implement dispatch_message using reflect_message
Nov 14, 2024
e8aaede
Merge pull request #103 from microavia/v1-reflection
DrTon Nov 18, 2024
92271d4
refactor: reformat code a bit
Nov 21, 2024
328c2e2
fix: fixed integration tests
Nov 21, 2024
3bbfb4b
refactor: modular nested converter poc
Nov 21, 2024
db9f882
refactor: fix types
Nov 22, 2024
6c15ee1
test: fix nested converter tests
Nov 22, 2024
ec8a57a
refacotr: clean code a bit
Nov 22, 2024
211f3fd
test: parse type tests
Nov 22, 2024
76f32f3
refactor: fix eol
Nov 22, 2024
059f0be
fix: fix cpp test
Nov 22, 2024
43da1ec
refactor: refactor code a bit
Nov 22, 2024
495d0a5
refactor: recursice converter init
Nov 22, 2024
2ec6a2a
test: update tests for new codec format
Nov 22, 2024
fdc44ab
refactor: move base converters to separated dir
Nov 22, 2024
d593fbc
refactor: fix benchmark
Nov 22, 2024
ffb7965
refactor: adjust performance
Nov 22, 2024
e350d88
refactor: fix string parsing perf problems
Nov 25, 2024
a6d11d5
refactor: optimize typed array size function
Nov 25, 2024
4939c63
refactor: add coverage tools
Nov 25, 2024
35602e7
test: setup coverage for the js messgen project
Nov 25, 2024
e879301
refactor: protocol -> protocols
Nov 25, 2024
b59f3b4
refactor: fix code style
Nov 25, 2024
b7c7681
Merge pull request #106 from microavia/feat/js-optimizations
khitrind Nov 26, 2024
52fd1ad
feat: messgen as js lib
Nov 27, 2024
b5f961b
feat: add eslint, types checks
Nov 27, 2024
9b624b9
feat: check types with eslint
Nov 27, 2024
7f8d168
refactor: fix eol for package json
Nov 27, 2024
5d2512d
chore: fix lint and eof for tsup
Nov 27, 2024
eadbfd1
Add useful attributes to serialization functions
Nov 28, 2024
2826a17
Avoid generating param name if unused
Nov 28, 2024
98753ad
Better formatting
Nov 28, 2024
5b2d63c
Ensure function signatures are one line
lachem Nov 28, 2024
eef30b0
Update cpp_generator.py
lachem Nov 28, 2024
ab4b8ac
Add useful concepts to messgen
Nov 28, 2024
56b9530
chore: add ci check for v1 branch
Nov 28, 2024
567439f
fix: fix eol for tsconfig file
Nov 28, 2024
133ab51
Merge pull request #107 from microavia/feat/js-port-as-npm-lib
khitrind Nov 28, 2024
fa54042
Merge pull request #109 from microavia/v1-concepts
DrTon Nov 28, 2024
5aa0124
Merge pull request #108 from microavia/v1-missing-attributes
DrTon Nov 28, 2024
dbb0913
feat: export types from js port
Nov 29, 2024
8728b6b
chore: package version bump
Nov 29, 2024
109a311
refactor: fix struct bench lint
Nov 29, 2024
98b43f3
Make message concepts for cv qualified types
Dec 3, 2024
7e0c8c5
feat: simplify creating custom structs
Dec 4, 2024
0800701
feat: simplify creating custom structs
Dec 4, 2024
c12424b
chore: version bump
Dec 4, 2024
1320373
Merge pull request #110 from microavia/v1_improve_concepts
DrTon Dec 4, 2024
59932d5
Use dataclass for parser results
Dec 4, 2024
0991cc7
Extract model to separate file
Dec 4, 2024
ec7cc0b
Improve parser readability #1 of 2
Dec 4, 2024
3e3cb72
Improve parser readability #2 of 2
Dec 4, 2024
62dd92a
Adapt generator to the new messgen model
Dec 4, 2024
a821da0
Use dataclass for enum values
Dec 4, 2024
6510d47
Improve class name qualification
Dec 4, 2024
fe8d45d
Fix empty namespace generation
Dec 4, 2024
c02d0e6
Define and generate new protocols
Dec 4, 2024
2a7627a
Fix reflect_message switch cases
Dec 4, 2024
919d232
Adopt cmake files to the new approach
Dec 4, 2024
8c2237e
Make tests compile and run (still fail)
Dec 4, 2024
302cb62
Remove whitespace from schema
Dec 4, 2024
8002563
Update concepts to make tests pass
Dec 4, 2024
ee54456
Extract concepts to separate files
Dec 4, 2024
004ec15
Remove spare protocols.py
Dec 4, 2024
507d5db
Use an enumeration for the type_class
Dec 5, 2024
c12535a
Use new messgen model in dynamic.py
Dec 5, 2024
4de7118
Remove spare imports
Dec 5, 2024
953614d
Fix generate_serialized_data test
Dec 5, 2024
5262fac
Fix remaining python unit tests
Dec 5, 2024
500c848
Build both debug and release in CI
Dec 5, 2024
2e8648c
Improve formatting of cpp CI
Dec 5, 2024
740d4f0
Improve cmake integration helper
Dec 5, 2024
2e07ccc
Move protocols and types into tests/data dir
Dec 5, 2024
ab901a9
Move serialized_data to tests/data/serialized
Dec 5, 2024
6315317
feat: set protocol by default
Dec 5, 2024
5d99544
Fix commandline params in js tests
Dec 5, 2024
ccc155d
Prepend data in messgen invoke command
Dec 5, 2024
13d921d
Update json_generator, improve validation
Dec 5, 2024
01f6080
Disable js CI workflow for now
Dec 5, 2024
5e07b9e
Improve comment
Dec 5, 2024
ab95af6
Improve code quality
Dec 5, 2024
98d3ff0
Refactor unit tests to proper python tests
Dec 5, 2024
d5a2b6a
Enable python tests in CI
Dec 5, 2024
4edc589
Disable js tests like there was no tomorrow
Dec 5, 2024
eb81623
Add empty requirements.txt
Dec 5, 2024
0e1b20c
Add pytest to requirements.txt
Dec 5, 2024
e7563c7
Add yaml to requirements.txt
Dec 5, 2024
1367054
yaml package is called pyyaml
Dec 5, 2024
3087309
Fix mypy errors #1 of n
Dec 5, 2024
2bfdd23
Fix mypy errors #2 of n
Dec 5, 2024
ad89c76
Fix mypy errors #3 of n
Dec 6, 2024
290a202
Fix mypy errors #4 of 4
Dec 6, 2024
3436409
Add mypy check for python workflow in git CI
Dec 6, 2024
abe0a51
Install pyyaml type hints
Dec 6, 2024
e52ba23
Update C++ keywords list
Dec 6, 2024
220b3c5
Allow dependend types to be unqualified
Dec 6, 2024
59701e3
Fix status print in main generator
Dec 6, 2024
0702591
Improve path handling in cmake integration
Dec 6, 2024
80e4c2c
Change input protocol format to path:protocol
Dec 6, 2024
19bcd99
Add nested types tests. Improve cmake and concepts.
Dec 6, 2024
fbccaa9
Make schema raw string more robust
Dec 6, 2024
e99f2df
Generate protocol as struct and not namespace
Dec 6, 2024
ac98990
Fix return type of split_last_name
lachem Dec 7, 2024
5ef6844
Fix return type of parse protocol
lachem Dec 7, 2024
8f7a890
Merge pull request #111 from microavia/v1-separate-types
DrTon Dec 7, 2024
aaf78ef
refactor: revert struct parser
Dec 11, 2024
8ceaa1f
Merge pull request #112 from microavia/feat/js_port_types
khitrind Dec 11, 2024
a223924
feat: support separated types generation for ts
Dec 11, 2024
8ff1c0c
feat: support separated protocols generation for ts
Dec 11, 2024
d9a9419
fix: fix mypy error
Dec 12, 2024
0795bb9
feat: combine protocols and types
Dec 12, 2024
1159e46
fix: fix node generate types scripts
Dec 12, 2024
79f1dc2
feat: support separated types for codec
Dec 12, 2024
319086e
test: fix js pipeline test
Dec 12, 2024
0df41a1
refactor: refactor js protocol a bit
Dec 12, 2024
3280435
Merge pull request #115 from microavia/feat/separate-types-js
khitrind Dec 12, 2024
6162b72
Update readme file following type/proto split (#113)
lachem Dec 12, 2024
061a36c
Validate in messgen-generate to avoid duplication
Dec 9, 2024
fa1feb7
Make cpp-generator conformant with clang format
Dec 10, 2024
b6e5750
Add type validation to avoid clashing hashes
Dec 10, 2024
cf985d8
Introduce type hashing in the model
Dec 10, 2024
c3b4845
Fix serialization / deserialization in dynamic
Dec 10, 2024
a22c670
Improve exception info in dynamic protocol
Dec 10, 2024
8e98541
Expose converter and type_hash
Dec 10, 2024
860c985
Introduce convenience overloads for serialization
Dec 10, 2024
a957f9b
Return type_hash on serialization / deserialization
Dec 10, 2024
e2b99a3
Improve converters to allow external usage
Dec 11, 2024
4e315ab
Improve naming in dynamic port
Dec 11, 2024
bc32333
Do not return type_hash from proto serialization
Dec 12, 2024
5648b0d
Add message descriptions to test protos
Dec 16, 2024
e3feb29
Introduce protocol message model type
Dec 16, 2024
42980c9
Seems members_of can be consteval
Dec 16, 2024
c9bce28
Remove unused code
Dec 16, 2024
45aa8c9
Rename member to member_variable
Dec 16, 2024
773c919
Cpp: generate msg structs instead of TYPE_ID var
Dec 16, 2024
00d0e9a
Remove spare print
Dec 16, 2024
94cdc9d
Make protocol message inherit from types
Dec 16, 2024
9ded34c
Better template parameter names
Dec 16, 2024
59cd0de
Test base aggregate initialization
Dec 16, 2024
91a98a9
Hash in dec is more useful
Dec 16, 2024
58a6b9f
Make concept stick with clang
Dec 16, 2024
c35d3a3
Make concept stick with clang
Dec 16, 2024
fcfaef1
Make message aliases more distinct
Dec 16, 2024
c39d2b2
Dispatch message since it inherits from type
Dec 16, 2024
ea9453b
Further improve message concept
Dec 16, 2024
c447316
Add protocol serialization fix and test
Dec 17, 2024
db8e934
Initialized base subobject explicitely
Dec 17, 2024
5321568
Remove double braces around scalar init
Dec 17, 2024
ce7b29c
Add MessageSerializer for more ergonomic API
Dec 17, 2024
2be9f08
TYPE_ID -> MESSAGE_ID in proto
Dec 17, 2024
ee2511a
Better names for dynamic.Codec member functions
Dec 17, 2024
3fcbf7f
Hash in dec is more debuggable
Dec 17, 2024
5cd89c6
More type annotations
Dec 18, 2024
a898471
Rename TypeSerializer to TypeConverter
Dec 18, 2024
6aebd71
Add Makefile
Dec 18, 2024
597280c
Update README
Dec 18, 2024
d441877
Consisent yaml formatting in README
lachem Dec 18, 2024
5426bc4
Fix merge issue
Dec 18, 2024
e630c80
Type ID -> Message ID in readme
Dec 18, 2024
b818537
Fix ts_generator
Dec 18, 2024
4922c95
Restore json generator
Dec 18, 2024
a03446a
Js will be updated in a separate PR
Dec 18, 2024
2a7ae74
Restore .gitignore
lachem Dec 18, 2024
dc94c0d
Merge pull request #114 from microavia/v1-separate-types
DrTon Dec 18, 2024
f376f66
Clarify header in README
lachem Dec 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/actions/node/action.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
runs:
using: "composite"
steps:

- name: Setup node
uses: actions/setup-node@v3
with:
node-version: 16
node-version: 22.11.0
registry-url: https://registry.npmjs.org/

- name: Cache node modules
Expand Down
35 changes: 26 additions & 9 deletions .github/workflows/cpp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,35 @@ name: C++ CI

on:
push:
branches: [ "master" ]
branches: ["master", "v1"]
pull_request:
branches: [ "master" ]
branches: ["master", "v1"]

jobs:
build:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Install gtest manually
run: sudo apt-get install libgtest-dev && cd /usr/src/gtest && sudo cmake CMakeLists.txt && sudo make && sudo cp lib/*.a /usr/lib && sudo ln -s /usr/lib/libgtest.a /usr/local/lib/libgtest.a && sudo ln -s /usr/lib/libgtest_main.a /usr/local/lib/libgtest_main.a
- name: Run tests
run: cd tests && make all
- uses: actions/checkout@v3

- uses: awalsh128/cache-apt-pkgs-action@latest
with:
packages: libgtest-dev cmake ninja-build gcc g++
version: 1.0

- name: Build Debug
run: |
mkdir -p build_debug
cmake -Bbuild_debug -S. -G Ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DCMAKE_BUILD_TYPE=Debug
cmake --build build_debug

- name: Test Debug
run: ctest --test-dir build_debug/tests/ --output-on-failure

- name: Build Release
run: |
mkdir -p build_release
cmake -Bbuild_release -S. -G Ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DCMAKE_BUILD_TYPE=Release
cmake --build build_release

- name: Test Release
run: ctest --test-dir build_release/tests/ --output-on-failure
17 changes: 13 additions & 4 deletions .github/workflows/js.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,13 @@ name: JS CI

on:
push:
branches: [ master ]
branches: [master, "v1"]
pull_request:
branches: [ master ]
branches: [master, "v1"]

jobs:
build:
if: false # disable the entire workflow
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
Expand All @@ -18,10 +19,18 @@ jobs:
working-directory: ./port/js
run: npm ci --prefer-offline --no-audit

- name: Run lint
working-directory: ./port/js
run: npm run check:lint

- name: Run type check
working-directory: ./port/js
run: npm run check:types

- name: Run tests
working-directory: ./port/js
run: npm test
run: npm run test

- name: Build
working-directory: ./port/js
run: npm run build
run: npm run build
33 changes: 33 additions & 0 deletions .github/workflows/python.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Python package

on:
push:
branches: ["master", "v1"]
pull_request:
branches: ["master", "v1"]

jobs:
test:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v2

- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.x'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt

- name: Run tests
run: |
python -m pytest

- name: Check mypy
run: |
python -m mypy .
9 changes: 5 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
# IntelliJ project files
.idea
.vscode
.run

# Build files
.cache
*.pyc
*.pyd
cmake-build-*/
build
Testing

# Log files
dump.txt
Expand All @@ -14,7 +19,3 @@ dump.txt
tests/cpp/messgen
.done.*

tests/js/tmp
tests/js/coverage
tests/js/messgen
tests/js/MessgenHelper.ts
9 changes: 9 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
cmake_minimum_required(VERSION 3.5)

project(messgen)

set(CMAKE_CXX_STANDARD 20)

include("integrations/cmake/messgen.cmake")

add_subdirectory(tests)
22 changes: 22 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
ROOT_DIR ?= $(realpath $(PWD))
BUILD_DIR ?= $(ROOT_DIR)/build
BUILD_TYPE ?= Debug

all: check

configure:
cmake -B${BUILD_DIR} -S. -G Ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DCMAKE_BUILD_TYPE=${BUILD_TYPE}

build: configure
cmake --build ${BUILD_DIR}

test: build
ctest --test-dir ${BUILD_DIR}/tests/ --output-on-failure
python3 -m pytest .

check: test
python3 -m mypy .

clean:
rm -rf ${BUILD_DIR}

Loading
Loading