From 00d5e56aa05d79b9c74afeb1117b5d37134275e6 Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Wed, 23 Oct 2024 13:36:22 -0400 Subject: [PATCH 001/105] Initial draft of MEX best practice --- MEX.md | 162 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 MEX.md diff --git a/MEX.md b/MEX.md new file mode 100644 index 0000000..3ef32e1 --- /dev/null +++ b/MEX.md @@ -0,0 +1,162 @@ +# MATLAB MEX Best Practice + +[CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) + +Welcome to the MATLAB® MEX Best Practice guide. This document provides a streamlined approach to integrating MEX files into your MATLAB toolbox projects, aligning with our existing [MATLAB Toolbox Best Practices](README.md). MEX files provide a mechanism for you to call a C or C++ function from within MATLAB. This guide will help you maintain a consistent and efficient workflow. Let's dive in! + +## Overview + +MEX files allow you to call C, C++, or Fortran code directly from MATLAB, offering access to the power of C and C++. However, managing these files can be complex. This guide will help you integrate MEX files effectively in both development and production environments. + +To make it easier to follow, we’ve created a fictitious toolbox for doing basic arithmetic: The Arithmetic Toolbox on [GitHub](https://github.com/mathworks/arithmetic). We’ll use this throughout to show how to apply these best practices. + +## Key Concepts + +- **MEX Files**: These are functions written in C, C++, or Fortran, compiled to be callable from MATLAB. See the [MEX documentation](https://www.mathworks.com/help/matlab/cpp-mex-file-applications.html) for more information. +- **`buildtool`**: MATLAB's tool for automating build processes, including MEX file compilation. See the [`buildtool` documentation](https://www.mathworks.com/help/matlab/ref/buildtool.html) for more information. +- **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. + +## MEX Source Files + +Put your MEX source files in a language-specific directory at the root level of your project, such as `cpp`. This structure keeps your code organized and makes it easier to manage multiple programming languages. + +Our example toolbox has two MEX functions: `addMex` and `subtractMex` which are called by the MATLAB functions `add.m` and `subtract.m`. + +``` text +arithmetic/ +├───toolbox/ +| ├───add.m +| └───subtract.m +├───cpp/ +│ ├───addMex/ +│ │ ├───firstFile.cpp +│ │ └───secondFile.cpp +│ └───subtractMex/ +│ └───subtract.cpp +├───arithmetic.prj +└───buildfile.m +``` + +## Building MEX Files +- **Tools**: Use MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b, to automate the MEX build process. The [`matlab.buildtool.tasks.MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html), introduced in R2024a, automates the compilation, ensuring consistency across environments. If you need support in earlier releases, use the [`mex` command](https://www.mathworks.com/help/matlab/ref/mex.html). +- **Compiled Binaries**: Place compiled MEX binaries in the `toolbox/private` folder. This keeps them internal and reduces risk from unhandled inputs, as MEX functions should not be directly accessible to end users. + +Our example toolbox shows the built mex functions for several platforms: + +``` text +arithmetic/ +: +├───toolbox/ +| ├───add.m +| ├───subtract.m +| └───private/ +| ├───addMex.mexw64 (derived) +| ├───addMex.mexa64 (derived) +| ├───addMex.mexmaca64 (derived) +| ├───subtractMex.mexw64 (derived) +| ├───subtractMex.mexa64 (derived) +| └───subtractMex.mexmaca64 (derived) +├───cpp/ +│ ├───addMex/ +│ │ ├───firstFile.cpp +│ │ └───secondFile.cpp +│ └───subtractMex/ +│ └───subtract.cpp +├───arithmetic.prj +└───buildfile.m +``` + +Example buildfile.m: +``` matlab +TBD +``` + +## Incorporating External Libraries + If your MEX function uses external libraries, check the binaries into the repository under a `libraries` directory with subdirectories for different platforms. For more advanced scenarios, consider using tools like [Conan](https://conan.io/) to manage dependencies. During the build, copy necessary libraries into the `toolbox/private` folder to avoid managing path issues on Windows, or `LD_LIBRARY_PATH` issues on Linux. + +Our example toolbox adds a library called `complex` to the `subtractMex` function: + +``` text +arithmetic/ +: +├───toolbox/ +| ├───add.m +| ├───subtract.m +| └───private/ +| ├───addMex.mexw64 (derived) +| ├───addMex.mexa64 (derived) +| ├───addMex.mexmaca64 (derived) +| ├───subtractMex.mexw64 (derived) +| ├───subtractMex.mexa64 (derived) +| └───subtractMex.mexmaca64 (derived) +├───cpp/ +│ ├───addMex/ +│ │ ├───firstFile.cpp +│ │ └───secondFile.cpp +│ └───subtractMex/ +│ ├───subtract.cpp +| └───libraries +| ├───complex.hpp +| ├───glnxa64 +| | └───libcomplex.so +| ├───maci64 +| | └───libcomplex.dylib +| └───win64 +| └───complex.dll +├───arithmetic.prj +└───buildfile.m +``` + +## Testing + +Write tests for the MATLAB layer in the `tests/` folder. MEX functionality is indirectly tested through MATLAB scripts: + +Our example toolbox adds `testAdd.m` and `testSubtract.m`to validate the functionality of `add.m` and `subtract.m` which, in turn, use `addMex` and `subtractMex`. + +``` text +arithmetic/ +: +├───toolbox/ +| ├───add.m +| ├───subtract.m +| └───private/ +| ├───addMex.mexw64 (derived) +| ├───addMex.mexa64 (derived) +| ├───addMex.mexmaca64 (derived) +| ├───subtractMex.mexw64 (derived) +| ├───subtractMex.mexa64 (derived) +| └───subtractMex.mexmaca64 (derived) +├───cpp/ +│ ├───addMex/ +│ │ ├───firstFile.cpp +│ │ └───secondFile.cpp +│ └───subtractMex/ +│ ├───subtract.cpp +| └───libraries +| ├───complex.hpp +| ├───glnxa64 +| | └───libcomplex.so +| ├───maci64 +| | └───libcomplex.dylib +| └───win64 +| └───complex.dll +├───tests/ +| ├───testAdd.m +| └───testSubstract.m +├───arithmetic.prj +└───buildfile.m +``` + +## Using GitHub Actions to Automate Builds +**TBD** + +## Conclusion + +Following these best practices ensures a robust, maintainable, and user-friendly MATLAB toolbox that leverages the power of MEX files. By organizing your project structure, automating builds, and managing dependencies effectively, you can focus on delivering high-performance solutions to your users. + +For more details, suggestions, or to contribute, feel free to [open an issue](https://github.com/mathworks/toolboxdesign/issues/new/choose). + +--- +[![CC-BY-4.0](images/cc-by-40.png)](https://creativecommons.org/licenses/by/4.0/) + +Copyright © 2023-2024, The MathWorks, Inc. From 7fc50f8a6b2fc7cfc9d84bffec7c790e9466cdc4 Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Wed, 23 Oct 2024 13:41:27 -0400 Subject: [PATCH 002/105] Add badges and under construction sign --- MEX.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index 3ef32e1..4f0736c 100644 --- a/MEX.md +++ b/MEX.md @@ -1,6 +1,9 @@ # MATLAB MEX Best Practice -[CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) +|:exclamation: This page is under construction and review | +|----------------------------------------------------------| + +![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) Welcome to the MATLAB® MEX Best Practice guide. This document provides a streamlined approach to integrating MEX files into your MATLAB toolbox projects, aligning with our existing [MATLAB Toolbox Best Practices](README.md). MEX files provide a mechanism for you to call a C or C++ function from within MATLAB. This guide will help you maintain a consistent and efficient workflow. Let's dive in! From be7f298e46a8598219144101c53345b72519a0ee Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Wed, 23 Oct 2024 13:44:01 -0400 Subject: [PATCH 003/105] Use a little more standard markdown --- MEX.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/MEX.md b/MEX.md index 4f0736c..c9b7d48 100644 --- a/MEX.md +++ b/MEX.md @@ -1,7 +1,7 @@ # MATLAB MEX Best Practice -|:exclamation: This page is under construction and review | -|----------------------------------------------------------| +> [!IMPORTANT] +> This page is under construction and review ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) From 2569d247abe907d2f85a4805842374f534e46bfa Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Wed, 23 Oct 2024 14:02:32 -0400 Subject: [PATCH 004/105] Incorporate feedback --- MEX.md | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/MEX.md b/MEX.md index c9b7d48..791a507 100644 --- a/MEX.md +++ b/MEX.md @@ -5,13 +5,13 @@ ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) -Welcome to the MATLAB® MEX Best Practice guide. This document provides a streamlined approach to integrating MEX files into your MATLAB toolbox projects, aligning with our existing [MATLAB Toolbox Best Practices](README.md). MEX files provide a mechanism for you to call a C or C++ function from within MATLAB. This guide will help you maintain a consistent and efficient workflow. Let's dive in! +Welcome to the MATLAB® MEX Best Practice guide. This document offers a refined approach to seamlessly integrating MEX files into your MATLAB toolbox projects, complementing our established [MATLAB Toolbox Best Practices](README.md). MEX files enable you to harness the power of C or C++ functions within MATLAB, and this guide will equip you with the knowledge to maintain a consistent and efficient workflow. Let's explore the world of MEX integration! ## Overview -MEX files allow you to call C, C++, or Fortran code directly from MATLAB, offering access to the power of C and C++. However, managing these files can be complex. This guide will help you integrate MEX files effectively in both development and production environments. +MEX files bridge the gap between MATLAB and C, C++, or Fortran, allowing you to leverage the strengths of these languages directly within MATLAB. While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This organized structure enhances maintainability and makes it easier for others to understand and contribute to your project. -To make it easier to follow, we’ve created a fictitious toolbox for doing basic arithmetic: The Arithmetic Toolbox on [GitHub](https://github.com/mathworks/arithmetic). We’ll use this throughout to show how to apply these best practices. +To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. ## Key Concepts @@ -21,9 +21,9 @@ To make it easier to follow, we’ve created a fictitious toolbox for doing basi ## MEX Source Files -Put your MEX source files in a language-specific directory at the root level of your project, such as `cpp`. This structure keeps your code organized and makes it easier to manage multiple programming languages. +Organize your MEX source files in language-specific directories at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. -Our example toolbox has two MEX functions: `addMex` and `subtractMex` which are called by the MATLAB functions `add.m` and `subtract.m`. +Our Arithmetic Toolbox example features two MEX functions: `addMex` and `subtractMex`, which are called by the MATLAB functions `add.m` and `subtract.m`. ``` text arithmetic/ @@ -42,7 +42,7 @@ arithmetic/ ## Building MEX Files - **Tools**: Use MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b, to automate the MEX build process. The [`matlab.buildtool.tasks.MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html), introduced in R2024a, automates the compilation, ensuring consistency across environments. If you need support in earlier releases, use the [`mex` command](https://www.mathworks.com/help/matlab/ref/mex.html). -- **Compiled Binaries**: Place compiled MEX binaries in the `toolbox/private` folder. This keeps them internal and reduces risk from unhandled inputs, as MEX functions should not be directly accessible to end users. +- **Compiled Binaries**: Store compiled MEX binaries in the `toolbox/private` folder. This approach maintains internal accessibility while mitigating the risk of crashes from unhandled inputs by preventing direct user access to MEX functions. Our example toolbox shows the built mex functions for several platforms: @@ -74,8 +74,11 @@ Example buildfile.m: TBD ``` + ## Incorporating External Libraries - If your MEX function uses external libraries, check the binaries into the repository under a `libraries` directory with subdirectories for different platforms. For more advanced scenarios, consider using tools like [Conan](https://conan.io/) to manage dependencies. During the build, copy necessary libraries into the `toolbox/private` folder to avoid managing path issues on Windows, or `LD_LIBRARY_PATH` issues on Linux. +* When your MEX function relies on external libraries, store the binaries in a `libraries` directory with platform-specific subdirectories, as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. +* For projects with complex dependencies, consider adopting dependency management tools like [Conan](https://conan.io/) which can significantly simplify library management across different platforms. +* During the build process, copy the external libraries into the `toolbox/private` folder to circumvent path management issues on Windows and `LD_LIBRARY_PATH` concerns on Linux. Our example toolbox adds a library called `complex` to the `subtractMex` function: @@ -112,7 +115,7 @@ arithmetic/ ## Testing -Write tests for the MATLAB layer in the `tests/` folder. MEX functionality is indirectly tested through MATLAB scripts: +Develop tests for the MATLAB layer in the `tests/` folder. While MEX functionality is indirectly tested through MATLAB scripts, this approach ensures comprehensive validation of your toolbox's capabilities. Our example toolbox adds `testAdd.m` and `testSubtract.m`to validate the functionality of `add.m` and `subtract.m` which, in turn, use `addMex` and `subtractMex`. @@ -150,14 +153,15 @@ arithmetic/ └───buildfile.m ``` -## Using GitHub Actions to Automate Builds + +## Automating Builds with GitHub Actions **TBD** ## Conclusion -Following these best practices ensures a robust, maintainable, and user-friendly MATLAB toolbox that leverages the power of MEX files. By organizing your project structure, automating builds, and managing dependencies effectively, you can focus on delivering high-performance solutions to your users. +By following to these best practices, you'll create a robust, maintainable, and user-friendly MATLAB toolbox that harnesses the full potential of MEX files. Through effective project structure organization, build automation, and dependency management, you can focus on delivering great solutions to your users. -For more details, suggestions, or to contribute, feel free to [open an issue](https://github.com/mathworks/toolboxdesign/issues/new/choose). +We welcome your input! For further details, suggestions, or to contribute, please [open an issue](https://github.com/mathworks/toolboxdesign/issues/new/choose). --- [![CC-BY-4.0](images/cc-by-40.png)](https://creativecommons.org/licenses/by/4.0/) From fe2ce4b388428f8a7e4638f2170b8a8b2938d387 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 14 Apr 2025 14:43:01 +0530 Subject: [PATCH 005/105] Added notes on toolbox/private --- MEX.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/MEX.md b/MEX.md index 791a507..8f93ea2 100644 --- a/MEX.md +++ b/MEX.md @@ -5,7 +5,7 @@ ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) -Welcome to the MATLAB® MEX Best Practice guide. This document offers a refined approach to seamlessly integrating MEX files into your MATLAB toolbox projects, complementing our established [MATLAB Toolbox Best Practices](README.md). MEX files enable you to harness the power of C or C++ functions within MATLAB, and this guide will equip you with the knowledge to maintain a consistent and efficient workflow. Let's explore the world of MEX integration! +Welcome to the MATLAB® MEX Best Practice guide. This document offers a refined approach to seamlessly integrating MEX files into your MATLAB toolbox projects, complementing our established [MATLAB Toolbox Best Practices](README.md). MEX files enable you to harness the power of C /C++/ Fortran functions within MATLAB, and this guide will equip you with the knowledge to maintain a consistent and efficient workflow. Let's explore the world of MEX integration! ## Overview @@ -15,10 +15,16 @@ To illustrate these best practices, we've created a sample project: The Arithmet ## Key Concepts -- **MEX Files**: These are functions written in C, C++, or Fortran, compiled to be callable from MATLAB. See the [MEX documentation](https://www.mathworks.com/help/matlab/cpp-mex-file-applications.html) for more information. +- **MEX Source Files**: These are functions written in C, C++, or Fortran, compiled to be callable from MATLAB. See the [MEX documentation](https://www.mathworks.com/help/matlab/cpp-mex-file-applications.html) for more information. +- **MEX Functions**: +- **MEX Gateway Function**: +- **Run time binaries**: - **`buildtool`**: MATLAB's tool for automating build processes, including MEX file compilation. See the [`buildtool` documentation](https://www.mathworks.com/help/matlab/ref/buildtool.html) for more information. - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. +## Managing MEX functions and run time binaries +[MATLAB Toolbox Best Practices](README.md) advocates for placing the files needed by the user to run the toolbox under the `toolbox` folder. Contents of this folder is what shipped to the users. For a toolbox that uses MEX, we recommend the MEX functions to be placed under a `private` folder within the `toolbox` folder. The content of the `private` folder can be ignored from version control (add the private folder to the .gitignore) since MEX functions are derived artifacts. In addition to MEX functions, the `private` folder can also have other binary files that are required during runtime. + ## MEX Source Files Organize your MEX source files in language-specific directories at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. From 914c5218c62cf02082d0da91fe139d921362a5e0 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 14 Apr 2025 14:57:12 +0530 Subject: [PATCH 006/105] Added a bit more about private folder --- MEX.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/MEX.md b/MEX.md index 8f93ea2..caf936e 100644 --- a/MEX.md +++ b/MEX.md @@ -22,8 +22,10 @@ To illustrate these best practices, we've created a sample project: The Arithmet - **`buildtool`**: MATLAB's tool for automating build processes, including MEX file compilation. See the [`buildtool` documentation](https://www.mathworks.com/help/matlab/ref/buildtool.html) for more information. - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. -## Managing MEX functions and run time binaries -[MATLAB Toolbox Best Practices](README.md) advocates for placing the files needed by the user to run the toolbox under the `toolbox` folder. Contents of this folder is what shipped to the users. For a toolbox that uses MEX, we recommend the MEX functions to be placed under a `private` folder within the `toolbox` folder. The content of the `private` folder can be ignored from version control (add the private folder to the .gitignore) since MEX functions are derived artifacts. In addition to MEX functions, the `private` folder can also have other binary files that are required during runtime. +## MEX functions and runtime binaries +[MATLAB Toolbox Best Practices](README.md) advocates for placing the files needed by the user to run the toolbox under the `toolbox` folder. Contents of this folder is what gets shipped to the user. For a toolbox that uses MEX, we recommend the MEX functions to be placed under a `private` folder within the `toolbox` folder. The content of the `private` folder can be ignored from version control (add the private folder to the .gitignore) since MEX functions are derived artifacts. In addition to MEX functions, the `private` folder can also have other binary files that are required at runtime. + +Improper use of MEX functions can lead to MATLAB crashing. An effective safeguard against this failure is to prevent direct access to MEX functions. We recommended calling the MEX function from a MATLAB script, by doing so the author can validate the inputs from a MATLAB script before passing it to the MEX function. Placing the the MEX functions within a private folder only facilitates this, since a private folder cannot be added to MATLAB path directly and only MATLAB scripts placed within the parent folder of a private folder can access its contents. ## MEX Source Files From 82e5b8e4283b1602b1710943f5198d76e1be2639 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 14 Apr 2025 15:01:43 +0530 Subject: [PATCH 007/105] Added a link to private functions --- MEX.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index caf936e..d3d7eef 100644 --- a/MEX.md +++ b/MEX.md @@ -23,7 +23,7 @@ To illustrate these best practices, we've created a sample project: The Arithmet - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. ## MEX functions and runtime binaries -[MATLAB Toolbox Best Practices](README.md) advocates for placing the files needed by the user to run the toolbox under the `toolbox` folder. Contents of this folder is what gets shipped to the user. For a toolbox that uses MEX, we recommend the MEX functions to be placed under a `private` folder within the `toolbox` folder. The content of the `private` folder can be ignored from version control (add the private folder to the .gitignore) since MEX functions are derived artifacts. In addition to MEX functions, the `private` folder can also have other binary files that are required at runtime. +[MATLAB Toolbox Best Practices](README.md) advocates for placing the files needed by the user to run the toolbox under the `toolbox` folder. Contents of this folder is what gets shipped to the user. For a toolbox that uses MEX, we recommend the MEX functions be placed under a `private` folder within the `toolbox` folder, thus making these MEX functions [Private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). The content of the `private` folder can be ignored from version control (add the private folder to the .gitignore) since MEX functions are derived artifacts. In addition to MEX functions, the `private` folder can also have other binary files that are required at runtime. Improper use of MEX functions can lead to MATLAB crashing. An effective safeguard against this failure is to prevent direct access to MEX functions. We recommended calling the MEX function from a MATLAB script, by doing so the author can validate the inputs from a MATLAB script before passing it to the MEX function. Placing the the MEX functions within a private folder only facilitates this, since a private folder cannot be added to MATLAB path directly and only MATLAB scripts placed within the parent folder of a private folder can access its contents. From 8119a6dd79c386f488d8d7b2aa4ebb672bc8eb57 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Wed, 16 Apr 2025 10:23:13 +0530 Subject: [PATCH 008/105] Noted on key concepts added --- MEX.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/MEX.md b/MEX.md index d3d7eef..fa0600a 100644 --- a/MEX.md +++ b/MEX.md @@ -16,17 +16,13 @@ To illustrate these best practices, we've created a sample project: The Arithmet ## Key Concepts - **MEX Source Files**: These are functions written in C, C++, or Fortran, compiled to be callable from MATLAB. See the [MEX documentation](https://www.mathworks.com/help/matlab/cpp-mex-file-applications.html) for more information. -- **MEX Functions**: +- **MEX Functions**: Platform dependent binaries that can be called directly from MATLAB, they behave almost like a MATLAB function. - **MEX Gateway Function**: -- **Run time binaries**: +- **Compile time binaries**: These are binaries need not be shipped to the customers, they are required only at compile time. Static libraries are a good example of these binaries. +- **Run time binaries**: These are platform dependent binaries, that the users need to run your toolbox, shared object libraries (.so files) in Linux and dynamic link libraries (.dll files) in Windows are good examples of run time binaries. - **`buildtool`**: MATLAB's tool for automating build processes, including MEX file compilation. See the [`buildtool` documentation](https://www.mathworks.com/help/matlab/ref/buildtool.html) for more information. - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. -## MEX functions and runtime binaries -[MATLAB Toolbox Best Practices](README.md) advocates for placing the files needed by the user to run the toolbox under the `toolbox` folder. Contents of this folder is what gets shipped to the user. For a toolbox that uses MEX, we recommend the MEX functions be placed under a `private` folder within the `toolbox` folder, thus making these MEX functions [Private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). The content of the `private` folder can be ignored from version control (add the private folder to the .gitignore) since MEX functions are derived artifacts. In addition to MEX functions, the `private` folder can also have other binary files that are required at runtime. - -Improper use of MEX functions can lead to MATLAB crashing. An effective safeguard against this failure is to prevent direct access to MEX functions. We recommended calling the MEX function from a MATLAB script, by doing so the author can validate the inputs from a MATLAB script before passing it to the MEX function. Placing the the MEX functions within a private folder only facilitates this, since a private folder cannot be added to MATLAB path directly and only MATLAB scripts placed within the parent folder of a private folder can access its contents. - ## MEX Source Files Organize your MEX source files in language-specific directories at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. @@ -47,6 +43,12 @@ arithmetic/ ├───arithmetic.prj └───buildfile.m ``` +## MEX functions and runtime binaries +[MATLAB Toolbox Best Practices](README.md) advocates for placing the files needed by the user to run the toolbox under the `toolbox` folder. Contents of this folder is what gets shipped to the user. For a toolbox that uses MEX, we recommend the MEX functions be placed under a `private` folder within the `toolbox` folder, thus making these MEX functions [Private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). The content of the `private` folder can be ignored from version control (add the private folder to the .gitignore) since MEX functions are derived artifacts. In addition to MEX functions, the `private` folder can also have other binary files that are required at runtime. + +Improper use of MEX functions can lead to MATLAB crashing. An effective safeguard against this failure is to prevent direct access to MEX functions. We recommended calling the MEX function from a MATLAB script, by doing so the author can validate the inputs from a MATLAB script before passing it to the MEX function. Placing the the MEX functions within a private folder only facilitates this, since a private folder cannot be added to MATLAB path directly and only MATLAB scripts placed within the parent folder of a private folder can access its contents. + + ## Building MEX Files - **Tools**: Use MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b, to automate the MEX build process. The [`matlab.buildtool.tasks.MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html), introduced in R2024a, automates the compilation, ensuring consistency across environments. If you need support in earlier releases, use the [`mex` command](https://www.mathworks.com/help/matlab/ref/mex.html). From ced478e60583287e4bf8a6d63c7a48a3152250cf Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Wed, 16 Apr 2025 10:56:58 +0530 Subject: [PATCH 009/105] Added notes for gateway function --- MEX.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/MEX.md b/MEX.md index fa0600a..9f037b9 100644 --- a/MEX.md +++ b/MEX.md @@ -14,11 +14,10 @@ MEX files bridge the gap between MATLAB and C, C++, or Fortran, allowing you to To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. ## Key Concepts - - **MEX Source Files**: These are functions written in C, C++, or Fortran, compiled to be callable from MATLAB. See the [MEX documentation](https://www.mathworks.com/help/matlab/cpp-mex-file-applications.html) for more information. - **MEX Functions**: Platform dependent binaries that can be called directly from MATLAB, they behave almost like a MATLAB function. -- **MEX Gateway Function**: -- **Compile time binaries**: These are binaries need not be shipped to the customers, they are required only at compile time. Static libraries are a good example of these binaries. +- **MEX Gateway Function**: Written in a MEX supported language, MATLAB accesses this function when a call is made to a MEX function. Each MEX function has only one gateway function written using the syntax of the source language. +- **Compile time binaries**: Static libraries are a good example of these binaries. You need not ship these binaries to your users. - **Run time binaries**: These are platform dependent binaries, that the users need to run your toolbox, shared object libraries (.so files) in Linux and dynamic link libraries (.dll files) in Windows are good examples of run time binaries. - **`buildtool`**: MATLAB's tool for automating build processes, including MEX file compilation. See the [`buildtool` documentation](https://www.mathworks.com/help/matlab/ref/buildtool.html) for more information. - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. From 0fbb623bf293c3aafeaa48ebeedbbd104e4403fc Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Wed, 16 Apr 2025 13:33:00 +0530 Subject: [PATCH 010/105] Added mexext notes for mexext --- MEX.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/MEX.md b/MEX.md index 9f037b9..a4a16df 100644 --- a/MEX.md +++ b/MEX.md @@ -15,16 +15,17 @@ To illustrate these best practices, we've created a sample project: The Arithmet ## Key Concepts - **MEX Source Files**: These are functions written in C, C++, or Fortran, compiled to be callable from MATLAB. See the [MEX documentation](https://www.mathworks.com/help/matlab/cpp-mex-file-applications.html) for more information. -- **MEX Functions**: Platform dependent binaries that can be called directly from MATLAB, they behave almost like a MATLAB function. -- **MEX Gateway Function**: Written in a MEX supported language, MATLAB accesses this function when a call is made to a MEX function. Each MEX function has only one gateway function written using the syntax of the source language. -- **Compile time binaries**: Static libraries are a good example of these binaries. You need not ship these binaries to your users. +- **MEX Functions**: Platform dependent binaries that can be called directly from MATLAB, they behave almost like a MATLAB function. MATLAB uses a platform dependent file extension for MEX functions, these extensions can be determined using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). +- **MEX Gateway Function**: Written in a MEX supported language, MATLAB accesses this function when a call is made to a MEX function. Each MEX function has only one gateway function. [MEX gateway functions written in C](https://www.mathworks.com/help/matlab/apiref/mexfunction.html) +- **Compile time binaries**: Static libraries are a good example of these binaries. These library binaries are required only at build time and you need not ship them to your users. - **Run time binaries**: These are platform dependent binaries, that the users need to run your toolbox, shared object libraries (.so files) in Linux and dynamic link libraries (.dll files) in Windows are good examples of run time binaries. - **`buildtool`**: MATLAB's tool for automating build processes, including MEX file compilation. See the [`buildtool` documentation](https://www.mathworks.com/help/matlab/ref/buildtool.html) for more information. - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. ## MEX Source Files -Organize your MEX source files in language-specific directories at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. +Organize your MEX source files in language-specific directories at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. We recommend using the `cpp` folder for both C and C++ source code. + Our Arithmetic Toolbox example features two MEX functions: `addMex` and `subtractMex`, which are called by the MATLAB functions `add.m` and `subtract.m`. From 7e6e02a7d68d7e5ba214b3d000604d036dc5589e Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Wed, 16 Apr 2025 14:30:22 +0530 Subject: [PATCH 011/105] Note on single source and multiple source mex functions --- MEX.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index a4a16df..511f78e 100644 --- a/MEX.md +++ b/MEX.md @@ -20,9 +20,15 @@ To illustrate these best practices, we've created a sample project: The Arithmet - **Compile time binaries**: Static libraries are a good example of these binaries. These library binaries are required only at build time and you need not ship them to your users. - **Run time binaries**: These are platform dependent binaries, that the users need to run your toolbox, shared object libraries (.so files) in Linux and dynamic link libraries (.dll files) in Windows are good examples of run time binaries. - **`buildtool`**: MATLAB's tool for automating build processes, including MEX file compilation. See the [`buildtool` documentation](https://www.mathworks.com/help/matlab/ref/buildtool.html) for more information. +- **MEX compiler**: Converts MEX source files into MEX functions. The MEX compiler can be accessed from MATLAB via the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, it can be invoked for the system terminal via the same command. - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. -## MEX Source Files +## Organizing MEX Source Files +MEX functions are classified into two groups: +1. Single source MEX function +2. Multiple source MEX function + +This classification is based on how many source files are required to generate a MEX function. For a single source MEX function, the source file implements a MEX gateway function and other functionalities that are called within the gateway function. A multiple source MEX function has one of its source files implement a gateway function while the other source files implement features and functionalities that are used in the gateway function. Organize your MEX source files in language-specific directories at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. We recommend using the `cpp` folder for both C and C++ source code. From 07922242eb50dcd0a07483c73a9b7fc00a6faaab Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Wed, 16 Apr 2025 15:40:48 +0530 Subject: [PATCH 012/105] Added notes for the mexfunctions folder --- MEX.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/MEX.md b/MEX.md index 511f78e..d0adecb 100644 --- a/MEX.md +++ b/MEX.md @@ -28,9 +28,13 @@ MEX functions are classified into two groups: 1. Single source MEX function 2. Multiple source MEX function -This classification is based on how many source files are required to generate a MEX function. For a single source MEX function, the source file implements a MEX gateway function and other functionalities that are called within the gateway function. A multiple source MEX function has one of its source files implement a gateway function while the other source files implement features and functionalities that are used in the gateway function. +This classification is based on how many source files are required to generate a MEX function. For a single source MEX function, the source file implements a MEX gateway function and other functionalities that are called within the gateway function. A multiple source MEX function on the other hand, has one of its source files implement a gateway function while the other source files implement features and functionalities that are used in the gateway function. Our folder structure for the MEX source files is largely based on this classification. -Organize your MEX source files in language-specific directories at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. We recommend using the `cpp` folder for both C and C++ source code. +Organize your MEX source files in language-specific directories at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. We recommend using the `cpp` folder at the root of the toolbox repo for both C and C++ source code. + +For C and C++ single source MEX functions, place the source code within the `mexfunctions` folder within the `cpp` folder. In case of Fortran, source code create a `fortran` folder within the toolbox root and move the Fortran single source MEX function within the `mexfunctions` folder. Note that a toolbox project can have the `cpp` and `fortran` folders if it has C, C++ and Fortran source code. + + For multiple source MEX functions create a folder for each MEX function and Our Arithmetic Toolbox example features two MEX functions: `addMex` and `subtractMex`, which are called by the MATLAB functions `add.m` and `subtract.m`. From c930145b3c58d7eb692560383bdc7f33d2c50296 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Wed, 16 Apr 2025 16:29:53 +0530 Subject: [PATCH 013/105] Notes on multiple source MEX functions --- MEX.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/MEX.md b/MEX.md index d0adecb..bc6d5ad 100644 --- a/MEX.md +++ b/MEX.md @@ -32,24 +32,24 @@ This classification is based on how many source files are required to generate a Organize your MEX source files in language-specific directories at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. We recommend using the `cpp` folder at the root of the toolbox repo for both C and C++ source code. -For C and C++ single source MEX functions, place the source code within the `mexfunctions` folder within the `cpp` folder. In case of Fortran, source code create a `fortran` folder within the toolbox root and move the Fortran single source MEX function within the `mexfunctions` folder. Note that a toolbox project can have the `cpp` and `fortran` folders if it has C, C++ and Fortran source code. +For C and C++ single source MEX functions, place the source code within the `mexfunctions` folder within the `cpp` folder. In case of Fortran, create a `fortran` folder within the toolbox root and move the source code for Fortran single source MEX function within the `mexfunctions` folder. Note that a toolbox project can have both `cpp` and `fortran` folders if it has MEX functions written using C/ C++ and Fortran. We also recommend the file names be suffixed with 'Mex' to indicate that the source files are to be compiled to MEX functions. - For multiple source MEX functions create a folder for each MEX function and +For multiple source MEX functions, create a folder for each MEX function. The name of the folder should be same as that of the MEX function. In addition, we recommend suffixing the folder name with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. Our Arithmetic Toolbox example features two MEX functions: `addMex` and `subtractMex`, which are called by the MATLAB functions `add.m` and `subtract.m`. ``` text arithmetic/ -├───toolbox/ -| ├───add.m -| └───subtract.m ├───cpp/ │ ├───addMex/ │ │ ├───firstFile.cpp │ │ └───secondFile.cpp -│ └───subtractMex/ -│ └───subtract.cpp +│ └───mexfunctions/ +│ └───subtractMex.cpp +├───toolbox/ +| ├───add.m +| └───subtract.m ├───arithmetic.prj └───buildfile.m ``` From f48fbadfe344d032e84cae13cef49733ae754e3a Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Wed, 16 Apr 2025 16:37:21 +0530 Subject: [PATCH 014/105] updated the folder structure --- MEX.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/MEX.md b/MEX.md index bc6d5ad..1e8e6a1 100644 --- a/MEX.md +++ b/MEX.md @@ -37,16 +37,16 @@ For C and C++ single source MEX functions, place the source code within the `mex For multiple source MEX functions, create a folder for each MEX function. The name of the folder should be same as that of the MEX function. In addition, we recommend suffixing the folder name with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. -Our Arithmetic Toolbox example features two MEX functions: `addMex` and `subtractMex`, which are called by the MATLAB functions `add.m` and `subtract.m`. +Our Arithmetic Toolbox example features two MEX functions: `addMex` and `subtractMex`, we have left the MEX function's extension to make the discussion platform agnostic. `addMEX` is implemented as a single source MEX function and placed under the `mexfunctions` within the `cpp` folder, while is substract is . which are called by the MATLAB functions `add.m` and `subtract.m`. ``` text arithmetic/ ├───cpp/ -│ ├───addMex/ -│ │ ├───firstFile.cpp -│ │ └───secondFile.cpp +│ ├───substractMex/ +│ │ ├───substract.cpp % Implements gateway function +│ │ └───substractImp.cpp % other features │ └───mexfunctions/ -│ └───subtractMex.cpp +│ └───addMex.cpp ├───toolbox/ | ├───add.m | └───subtract.m From 747b6d83432b08ea0362d3458205fb9fde9d5ee3 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Wed, 16 Apr 2025 16:43:29 +0530 Subject: [PATCH 015/105] Described the organization of files within cpp folder for the arithmetic toolbox --- MEX.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index 1e8e6a1..d142ca5 100644 --- a/MEX.md +++ b/MEX.md @@ -37,7 +37,7 @@ For C and C++ single source MEX functions, place the source code within the `mex For multiple source MEX functions, create a folder for each MEX function. The name of the folder should be same as that of the MEX function. In addition, we recommend suffixing the folder name with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. -Our Arithmetic Toolbox example features two MEX functions: `addMex` and `subtractMex`, we have left the MEX function's extension to make the discussion platform agnostic. `addMEX` is implemented as a single source MEX function and placed under the `mexfunctions` within the `cpp` folder, while is substract is . which are called by the MATLAB functions `add.m` and `subtract.m`. +Our Arithmetic Toolbox example features two MEX functions: `addMex` and `subtractMex`, we have left the MEX function's extension to make the discussion platform agnostic. `addMEX` is implemented as a single source MEX function and placed under the `mexfunctions` folder within the `cpp` folder, while `substractMEX` is implemented as a MEX function with multiple source and the source code that compiles to `substractMEX` is place within the `substractMex` under the `cpp` folder. `substract.cpp` implements the MEX gateway function and `substractImp.cpp` contains implementations that are required for the functionality of `substractMEX` function. ``` text arithmetic/ From bf983fa38c7ead03518f7ad77f68e0c477a14c3e Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Wed, 16 Apr 2025 17:02:27 +0530 Subject: [PATCH 016/105] Added content for organizing mex functions --- MEX.md | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/MEX.md b/MEX.md index d142ca5..fa0224d 100644 --- a/MEX.md +++ b/MEX.md @@ -53,15 +53,19 @@ arithmetic/ ├───arithmetic.prj └───buildfile.m ``` -## MEX functions and runtime binaries -[MATLAB Toolbox Best Practices](README.md) advocates for placing the files needed by the user to run the toolbox under the `toolbox` folder. Contents of this folder is what gets shipped to the user. For a toolbox that uses MEX, we recommend the MEX functions be placed under a `private` folder within the `toolbox` folder, thus making these MEX functions [Private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). The content of the `private` folder can be ignored from version control (add the private folder to the .gitignore) since MEX functions are derived artifacts. In addition to MEX functions, the `private` folder can also have other binary files that are required at runtime. +## Building MEX Functions +- **Tools**: Use MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b, to automate the MEX build process. The [`matlab.buildtool.tasks.MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html), introduced in R2024a, automates the compilation, ensuring consistency across environments. If you need support in earlier releases, use the [`mex` command](https://www.mathworks.com/help/matlab/ref/mex.html). -Improper use of MEX functions can lead to MATLAB crashing. An effective safeguard against this failure is to prevent direct access to MEX functions. We recommended calling the MEX function from a MATLAB script, by doing so the author can validate the inputs from a MATLAB script before passing it to the MEX function. Placing the the MEX functions within a private folder only facilitates this, since a private folder cannot be added to MATLAB path directly and only MATLAB scripts placed within the parent folder of a private folder can access its contents. +## Organizing MEX Functions +[MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder. Contents of this folder is what gets shipped to the user. For a toolbox that uses MEX, we recommend the MEX functions be placed under a `private` folder within the `toolbox` folder, making these MEX functions [Private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). +We recommend ignoring the MEX functions with `private` folder from version control (add the private folder to the `.gitignore`) since MEX functions are derived artifacts. + +Our motivation for placing the MEX functions with the `private` folder is to restrict the user from directly calling the MEX function. + +stems from the observation that improper use of MEX functions can lead to MATLAB crashing. By placing MEX function An effective safeguard against this failure is to prevent direct access to MEX functions. We recommended calling the MEX function from a MATLAB script, by doing so the author can validate the inputs from a MATLAB script before passing it to the MEX function. Placing the the MEX functions within a private folder only facilitates this, since a private folder cannot be added to MATLAB path directly and only MATLAB scripts placed within the parent folder of a private folder can access its contents. -## Building MEX Files -- **Tools**: Use MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b, to automate the MEX build process. The [`matlab.buildtool.tasks.MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html), introduced in R2024a, automates the compilation, ensuring consistency across environments. If you need support in earlier releases, use the [`mex` command](https://www.mathworks.com/help/matlab/ref/mex.html). - **Compiled Binaries**: Store compiled MEX binaries in the `toolbox/private` folder. This approach maintains internal accessibility while mitigating the risk of crashes from unhandled inputs by preventing direct user access to MEX functions. Our example toolbox shows the built mex functions for several platforms: @@ -69,6 +73,12 @@ Our example toolbox shows the built mex functions for several platforms: ``` text arithmetic/ : +├───cpp/ +│ ├───substractMex/ +│ │ ├───substract.cpp +│ │ └───substractImp.cpp +│ └───mexfunctions/ +│ └───addMex.cpp ├───toolbox/ | ├───add.m | ├───subtract.m @@ -79,12 +89,6 @@ arithmetic/ | ├───subtractMex.mexw64 (derived) | ├───subtractMex.mexa64 (derived) | └───subtractMex.mexmaca64 (derived) -├───cpp/ -│ ├───addMex/ -│ │ ├───firstFile.cpp -│ │ └───secondFile.cpp -│ └───subtractMex/ -│ └───subtract.cpp ├───arithmetic.prj └───buildfile.m ``` From cb92d20877bbf4ca85d0edeb73f90032748d7925 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Wed, 16 Apr 2025 17:11:08 +0530 Subject: [PATCH 017/105] Added notes for placing mexfunctions within private folder --- MEX.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/MEX.md b/MEX.md index fa0224d..844751f 100644 --- a/MEX.md +++ b/MEX.md @@ -60,13 +60,11 @@ arithmetic/ ## Organizing MEX Functions [MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder. Contents of this folder is what gets shipped to the user. For a toolbox that uses MEX, we recommend the MEX functions be placed under a `private` folder within the `toolbox` folder, making these MEX functions [Private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). -We recommend ignoring the MEX functions with `private` folder from version control (add the private folder to the `.gitignore`) since MEX functions are derived artifacts. - -Our motivation for placing the MEX functions with the `private` folder is to restrict the user from directly calling the MEX function. +Our motivation for placing the MEX functions with the `private` folder is to restrict the toolbox user from calling the MEX function directly. We recommend accessing the MEX functions always from a MATLAB script, this approach gives toolbox authors control over what gets passed as input to the MEX functions, their by elimination unexpected MATLAB crashes. -stems from the observation that improper use of MEX functions can lead to MATLAB crashing. By placing MEX function An effective safeguard against this failure is to prevent direct access to MEX functions. We recommended calling the MEX function from a MATLAB script, by doing so the author can validate the inputs from a MATLAB script before passing it to the MEX function. Placing the the MEX functions within a private folder only facilitates this, since a private folder cannot be added to MATLAB path directly and only MATLAB scripts placed within the parent folder of a private folder can access its contents. +We recommend ignoring the MEX functions with `private` folder from version control (add the private folder to the `.gitignore`) since MEX functions are derived artifacts. -- **Compiled Binaries**: Store compiled MEX binaries in the `toolbox/private` folder. This approach maintains internal accessibility while mitigating the risk of crashes from unhandled inputs by preventing direct user access to MEX functions. + Our example toolbox shows the built mex functions for several platforms: From 23a0ea6b7a945886b48081cafefce541ef3b0da9 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 17 Apr 2025 09:21:37 +0530 Subject: [PATCH 018/105] Added the build script --- MEX.md | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/MEX.md b/MEX.md index 844751f..231b163 100644 --- a/MEX.md +++ b/MEX.md @@ -58,9 +58,9 @@ arithmetic/ ## Organizing MEX Functions -[MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder. Contents of this folder is what gets shipped to the user. For a toolbox that uses MEX, we recommend the MEX functions be placed under a `private` folder within the `toolbox` folder, making these MEX functions [Private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). +[MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder, contents of this folder gets shipped to the user. For a toolbox that uses MEX, place the MEX functions under a `private` folder within the `toolbox` folder, thus making them private [Private functions](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). -Our motivation for placing the MEX functions with the `private` folder is to restrict the toolbox user from calling the MEX function directly. We recommend accessing the MEX functions always from a MATLAB script, this approach gives toolbox authors control over what gets passed as input to the MEX functions, their by elimination unexpected MATLAB crashes. +Our motivation for making MEX functions private is to restrict the toolbox user from calling the MEX function directly. We recommend accessing the MEX functions always from a MATLAB script, this approach gives toolbox authors control over what gets passed as input to the MEX functions, their by elimination unexpected MATLAB crashes. We recommend ignoring the MEX functions with `private` folder from version control (add the private folder to the `.gitignore`) since MEX functions are derived artifacts. @@ -91,9 +91,22 @@ arithmetic/ └───buildfile.m ``` -Example buildfile.m: +Here is a `buildfile` with a single task called `mex`, it compile the single source and multiple source MEX functions. Note that the `buildfile` was tested on Windows, with MATLAB R2025a. + ``` matlab -TBD +function plan = buildfile + +import matlab.buildtool.tasks.* +import matlab.buildtool.io.* + +% Create a plan +plan = buildplan(); + +% Compile all the .cpp files inside cpp/mexfunctions into MEX functions +mexSourceFiles = files(plan, fullfile("cpp", "mexfunctions", "*.cpp")); +plan("mex") = MexTask.forEachFile(mexSourceFiles, fullfile("toolbox","private")); + +end ``` From 51b6e58ebc1f353ed7fe5aa16f33e67b58977182 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 17 Apr 2025 09:23:45 +0530 Subject: [PATCH 019/105] Minor update to the buildfile --- MEX.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index 231b163..1d71e0b 100644 --- a/MEX.md +++ b/MEX.md @@ -104,7 +104,8 @@ plan = buildplan(); % Compile all the .cpp files inside cpp/mexfunctions into MEX functions mexSourceFiles = files(plan, fullfile("cpp", "mexfunctions", "*.cpp")); -plan("mex") = MexTask.forEachFile(mexSourceFiles, fullfile("toolbox","private")); +mexOutputFolder = fullfile("toolbox","private")l; +plan("mex") = MexTask.forEachFile(mexSourceFiles, mexOutputFolder); end ``` From 27a0c321101abd7479c679ebc0fe4820362fdbd1 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 09:39:45 +0530 Subject: [PATCH 020/105] Minor updates --- MEX.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/MEX.md b/MEX.md index 1d71e0b..2134b63 100644 --- a/MEX.md +++ b/MEX.md @@ -9,18 +9,18 @@ Welcome to the MATLAB® MEX Best Practice guide. This document offers a refin ## Overview -MEX files bridge the gap between MATLAB and C, C++, or Fortran, allowing you to leverage the strengths of these languages directly within MATLAB. While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This organized structure enhances maintainability and makes it easier for others to understand and contribute to your project. +MEX functions bridge the gap between MATLAB and C, C++, or Fortran, allowing you to leverage the strengths of these languages directly within MATLAB. While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This organized structure enhances maintainability and makes it easier for others to understand and contribute to your project. To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. ## Key Concepts - **MEX Source Files**: These are functions written in C, C++, or Fortran, compiled to be callable from MATLAB. See the [MEX documentation](https://www.mathworks.com/help/matlab/cpp-mex-file-applications.html) for more information. -- **MEX Functions**: Platform dependent binaries that can be called directly from MATLAB, they behave almost like a MATLAB function. MATLAB uses a platform dependent file extension for MEX functions, these extensions can be determined using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). -- **MEX Gateway Function**: Written in a MEX supported language, MATLAB accesses this function when a call is made to a MEX function. Each MEX function has only one gateway function. [MEX gateway functions written in C](https://www.mathworks.com/help/matlab/apiref/mexfunction.html) -- **Compile time binaries**: Static libraries are a good example of these binaries. These library binaries are required only at build time and you need not ship them to your users. -- **Run time binaries**: These are platform dependent binaries, that the users need to run your toolbox, shared object libraries (.so files) in Linux and dynamic link libraries (.dll files) in Windows are good examples of run time binaries. +- **MEX Functions**: Platform dependent binaries that can be called directly from MATLAB, they behave almost like a MATLAB function. MATLAB uses a platform dependent file extension for MEX functions, you can determine the MEX extension for your platform using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). +- **MEX Gateway Function**: Written in a MEX supported language, MATLAB accesses this function when a call is made to the MEX function. Each MEX function can have one gateway function only. [MEX gateway functions written in C](https://www.mathworks.com/help/matlab/apiref/mexfunction.html) +- **Compile time binaries**: Static libraries are a good examples of compile time binaries. These library binaries are required only at build time and you need not ship them to your users. +- **Run time binaries**: These are platform dependent binaries, that the users need to run your toolbox, shared object libraries (.so files) in Linux and dynamic link libraries (.dll files) in Windows are good examples. - **`buildtool`**: MATLAB's tool for automating build processes, including MEX file compilation. See the [`buildtool` documentation](https://www.mathworks.com/help/matlab/ref/buildtool.html) for more information. -- **MEX compiler**: Converts MEX source files into MEX functions. The MEX compiler can be accessed from MATLAB via the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, it can be invoked for the system terminal via the same command. +- **MEX compiler**: Converts MEX source files into MEX functions. The MEX compiler can be accessed from MATLAB via the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, it can be invoked for the system terminal via the same command. You can configure C/ C++/ Fortran compilers using the [mex -setup](https://www.mathworks.com/help/matlab/matlab_external/changing-default-compiler.html). - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. ## Organizing MEX Source Files @@ -91,7 +91,7 @@ arithmetic/ └───buildfile.m ``` -Here is a `buildfile` with a single task called `mex`, it compile the single source and multiple source MEX functions. Note that the `buildfile` was tested on Windows, with MATLAB R2025a. +Here is a `buildfile` with a single task called `mex`, it compile the single and multiple source MEX functions. Note that the `buildfile` was tested on Windows, with MATLAB R2025a. ``` matlab function plan = buildfile From 5f3dc60498a642f67cf9719aea7702f9edecda52 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 09:45:31 +0530 Subject: [PATCH 021/105] reorganized the mex source files section --- MEX.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/MEX.md b/MEX.md index 2134b63..43afc03 100644 --- a/MEX.md +++ b/MEX.md @@ -24,13 +24,14 @@ To illustrate these best practices, we've created a sample project: The Arithmet - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. ## Organizing MEX Source Files +Organize your MEX source files in language-specific folder at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. We recommend using the `cpp` folder at the root of the toolbox repo for both C and C++ source code. + MEX functions are classified into two groups: 1. Single source MEX function 2. Multiple source MEX function -This classification is based on how many source files are required to generate a MEX function. For a single source MEX function, the source file implements a MEX gateway function and other functionalities that are called within the gateway function. A multiple source MEX function on the other hand, has one of its source files implement a gateway function while the other source files implement features and functionalities that are used in the gateway function. Our folder structure for the MEX source files is largely based on this classification. +Our folder structure within the language-specific folder (e.g., `cpp`) for the MEX source files is largely based on this classification. For a single source MEX function, the source code file implements the MEX gateway function and other functionalities that are called within the gateway function. A multiple source MEX function on the other hand, has one of its source code files implement the gateway function while the other source files implement features and functionalities that are used in the gateway function. -Organize your MEX source files in language-specific directories at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. We recommend using the `cpp` folder at the root of the toolbox repo for both C and C++ source code. For C and C++ single source MEX functions, place the source code within the `mexfunctions` folder within the `cpp` folder. In case of Fortran, create a `fortran` folder within the toolbox root and move the source code for Fortran single source MEX function within the `mexfunctions` folder. Note that a toolbox project can have both `cpp` and `fortran` folders if it has MEX functions written using C/ C++ and Fortran. We also recommend the file names be suffixed with 'Mex' to indicate that the source files are to be compiled to MEX functions. From cce3e12547b3fc59bc6bf5b5c83f1f68213ad326 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 09:56:52 +0530 Subject: [PATCH 022/105] Added the reasoning for Mex suffix --- MEX.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index 43afc03..8bde349 100644 --- a/MEX.md +++ b/MEX.md @@ -33,7 +33,8 @@ MEX functions are classified into two groups: Our folder structure within the language-specific folder (e.g., `cpp`) for the MEX source files is largely based on this classification. For a single source MEX function, the source code file implements the MEX gateway function and other functionalities that are called within the gateway function. A multiple source MEX function on the other hand, has one of its source code files implement the gateway function while the other source files implement features and functionalities that are used in the gateway function. -For C and C++ single source MEX functions, place the source code within the `mexfunctions` folder within the `cpp` folder. In case of Fortran, create a `fortran` folder within the toolbox root and move the source code for Fortran single source MEX function within the `mexfunctions` folder. Note that a toolbox project can have both `cpp` and `fortran` folders if it has MEX functions written using C/ C++ and Fortran. We also recommend the file names be suffixed with 'Mex' to indicate that the source files are to be compiled to MEX functions. +For C and C++ single source MEX functions, place the source code within the `mexfunctions` folder within the `cpp` folder. In case of Fortran, create a `fortran` folder within the toolbox root and move the source code for Fortran single source MEX function within the `mexfunctions` folder. Note that a toolbox project can have both `cpp` and `fortran` folders if it has MEX functions written using C/ C++ and Fortran. We also recommend the names of the MEX functions to be same as their source file (of course with different extensions), this ensures traceability. Moreover, we also recommend the file names be suffixed with 'Mex', this acts as an indication that the function being called is not any MATLAB function but a MEX function. + For multiple source MEX functions, create a folder for each MEX function. The name of the folder should be same as that of the MEX function. In addition, we recommend suffixing the folder name with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. From aa9ceb93494740e09993d3682d6b43445eb82f4f Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 10:00:09 +0530 Subject: [PATCH 023/105] Added project root, to list of definitions --- MEX.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index 8bde349..b443fec 100644 --- a/MEX.md +++ b/MEX.md @@ -21,6 +21,7 @@ To illustrate these best practices, we've created a sample project: The Arithmet - **Run time binaries**: These are platform dependent binaries, that the users need to run your toolbox, shared object libraries (.so files) in Linux and dynamic link libraries (.dll files) in Windows are good examples. - **`buildtool`**: MATLAB's tool for automating build processes, including MEX file compilation. See the [`buildtool` documentation](https://www.mathworks.com/help/matlab/ref/buildtool.html) for more information. - **MEX compiler**: Converts MEX source files into MEX functions. The MEX compiler can be accessed from MATLAB via the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, it can be invoked for the system terminal via the same command. You can configure C/ C++/ Fortran compilers using the [mex -setup](https://www.mathworks.com/help/matlab/matlab_external/changing-default-compiler.html). +- **Project Root**: The folder under which the toolbox source code is organized. A git repository is initialized under this folder. - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. ## Organizing MEX Source Files @@ -36,7 +37,7 @@ Our folder structure within the language-specific folder (e.g., `cpp`) for the M For C and C++ single source MEX functions, place the source code within the `mexfunctions` folder within the `cpp` folder. In case of Fortran, create a `fortran` folder within the toolbox root and move the source code for Fortran single source MEX function within the `mexfunctions` folder. Note that a toolbox project can have both `cpp` and `fortran` folders if it has MEX functions written using C/ C++ and Fortran. We also recommend the names of the MEX functions to be same as their source file (of course with different extensions), this ensures traceability. Moreover, we also recommend the file names be suffixed with 'Mex', this acts as an indication that the function being called is not any MATLAB function but a MEX function. -For multiple source MEX functions, create a folder for each MEX function. The name of the folder should be same as that of the MEX function. In addition, we recommend suffixing the folder name with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. +For multiple source MEX functions, create a folder for each MEX function within the respective language folder within the project root. The name of the folder should be same as that of the MEX function. In addition, we recommend suffixing the folder name with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. Our Arithmetic Toolbox example features two MEX functions: `addMex` and `subtractMex`, we have left the MEX function's extension to make the discussion platform agnostic. `addMEX` is implemented as a single source MEX function and placed under the `mexfunctions` folder within the `cpp` folder, while `substractMEX` is implemented as a MEX function with multiple source and the source code that compiles to `substractMEX` is place within the `substractMex` under the `cpp` folder. `substract.cpp` implements the MEX gateway function and `substractImp.cpp` contains implementations that are required for the functionality of `substractMEX` function. From 7b0f30b63f34c654dae275821a962ae16d303c57 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 10:03:05 +0530 Subject: [PATCH 024/105] Added notes for multiple source mex. --- MEX.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/MEX.md b/MEX.md index b443fec..afcd965 100644 --- a/MEX.md +++ b/MEX.md @@ -33,11 +33,10 @@ MEX functions are classified into two groups: Our folder structure within the language-specific folder (e.g., `cpp`) for the MEX source files is largely based on this classification. For a single source MEX function, the source code file implements the MEX gateway function and other functionalities that are called within the gateway function. A multiple source MEX function on the other hand, has one of its source code files implement the gateway function while the other source files implement features and functionalities that are used in the gateway function. - For C and C++ single source MEX functions, place the source code within the `mexfunctions` folder within the `cpp` folder. In case of Fortran, create a `fortran` folder within the toolbox root and move the source code for Fortran single source MEX function within the `mexfunctions` folder. Note that a toolbox project can have both `cpp` and `fortran` folders if it has MEX functions written using C/ C++ and Fortran. We also recommend the names of the MEX functions to be same as their source file (of course with different extensions), this ensures traceability. Moreover, we also recommend the file names be suffixed with 'Mex', this acts as an indication that the function being called is not any MATLAB function but a MEX function. -For multiple source MEX functions, create a folder for each MEX function within the respective language folder within the project root. The name of the folder should be same as that of the MEX function. In addition, we recommend suffixing the folder name with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. +For multiple source MEX functions, create a folder for each MEX function within the respective language folder under project root. The name of the folder should be same as that of the MEX function that will be created out for source code with the folder. The folder name has to be suffixed with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. Our Arithmetic Toolbox example features two MEX functions: `addMex` and `subtractMex`, we have left the MEX function's extension to make the discussion platform agnostic. `addMEX` is implemented as a single source MEX function and placed under the `mexfunctions` folder within the `cpp` folder, while `substractMEX` is implemented as a MEX function with multiple source and the source code that compiles to `substractMEX` is place within the `substractMex` under the `cpp` folder. `substract.cpp` implements the MEX gateway function and `substractImp.cpp` contains implementations that are required for the functionality of `substractMEX` function. From b9fabe27ba750eefdf589e3db8b9b13067a40ea5 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 10:38:02 +0530 Subject: [PATCH 025/105] Refined the language for source code organization --- MEX.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/MEX.md b/MEX.md index afcd965..1d581bc 100644 --- a/MEX.md +++ b/MEX.md @@ -36,10 +36,10 @@ Our folder structure within the language-specific folder (e.g., `cpp`) for the M For C and C++ single source MEX functions, place the source code within the `mexfunctions` folder within the `cpp` folder. In case of Fortran, create a `fortran` folder within the toolbox root and move the source code for Fortran single source MEX function within the `mexfunctions` folder. Note that a toolbox project can have both `cpp` and `fortran` folders if it has MEX functions written using C/ C++ and Fortran. We also recommend the names of the MEX functions to be same as their source file (of course with different extensions), this ensures traceability. Moreover, we also recommend the file names be suffixed with 'Mex', this acts as an indication that the function being called is not any MATLAB function but a MEX function. -For multiple source MEX functions, create a folder for each MEX function within the respective language folder under project root. The name of the folder should be same as that of the MEX function that will be created out for source code with the folder. The folder name has to be suffixed with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. +For multiple source MEX functions, create a folder for each MEX function within the respective language folder. The name of the folder should be same as that of the MEX function that will be created out for source code with the folder. The folder name can be suffixed with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function, with the same name as the folder. -Our Arithmetic Toolbox example features two MEX functions: `addMex` and `subtractMex`, we have left the MEX function's extension to make the discussion platform agnostic. `addMEX` is implemented as a single source MEX function and placed under the `mexfunctions` folder within the `cpp` folder, while `substractMEX` is implemented as a MEX function with multiple source and the source code that compiles to `substractMEX` is place within the `substractMex` under the `cpp` folder. `substract.cpp` implements the MEX gateway function and `substractImp.cpp` contains implementations that are required for the functionality of `substractMEX` function. +Our [Arithmetic Toolbox]() example features two MEX functions: `addMex` and `subtractMex`. `addMex` is implemented as a single source MEX function and the source code is placed under the `mexfunctions` folder. On the other hand, `substractMex` is implemented as a multiple source MEX function with source code placed under the `substractMex` folder, `substract.cpp` implements the MEX gateway function for the `substractMex` MEX function and `substractImp.cpp` implementations functionalities that are used in `substract.cpp`. The organization of the MEX source files is shown below: ``` text arithmetic/ @@ -56,7 +56,7 @@ arithmetic/ └───buildfile.m ``` ## Building MEX Functions -- **Tools**: Use MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b, to automate the MEX build process. The [`matlab.buildtool.tasks.MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html), introduced in R2024a, automates the compilation, ensuring consistency across environments. If you need support in earlier releases, use the [`mex` command](https://www.mathworks.com/help/matlab/ref/mex.html). +- **Tools**: Use MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b, to automate the MEX build process. The [`matlab.buildtool.tasks.MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html), introduced in R2024a, automates the compilation, ensuring consistency across environments. If you need support in earlier releases, use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command directly. ## Organizing MEX Functions From 411a06a01a566a1ef8eb2dc88de8d936846668c2 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 10:52:17 +0530 Subject: [PATCH 026/105] Added notes for building mex functions --- MEX.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/MEX.md b/MEX.md index 1d581bc..63a5dab 100644 --- a/MEX.md +++ b/MEX.md @@ -62,13 +62,18 @@ arithmetic/ ## Organizing MEX Functions [MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder, contents of this folder gets shipped to the user. For a toolbox that uses MEX, place the MEX functions under a `private` folder within the `toolbox` folder, thus making them private [Private functions](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). -Our motivation for making MEX functions private is to restrict the toolbox user from calling the MEX function directly. We recommend accessing the MEX functions always from a MATLAB script, this approach gives toolbox authors control over what gets passed as input to the MEX functions, their by elimination unexpected MATLAB crashes. +Our motivation for making MEX functions private is to restrict the toolbox user from calling them directly. We recommend accessing the MEX functions always from a MATLAB script, this approach gives toolbox authors control over what gets passed as input to the MEX functions, their by elimination failures due to unhandled inputs. -We recommend ignoring the MEX functions with `private` folder from version control (add the private folder to the `.gitignore`) since MEX functions are derived artifacts. +We recommend ignoring the MEX functions from version control (add the `private` folder to the `.gitignore`) since MEX functions are derived artifacts. -Our example toolbox shows the built mex functions for several platforms: + + +The layout of the files with the `private` folder is shown below, for the sake of clarity we have shown the MEX functions for all the major platforms. Not that the GitHub repository for the Arithmetic toolbox will not contain the MEX functions since they are ignored by the version control system. You can create the MEX functions by running the following command from MATLAB terminal: +``` matlab +>> buildtool mex +``` ``` text arithmetic/ @@ -93,7 +98,7 @@ arithmetic/ └───buildfile.m ``` -Here is a `buildfile` with a single task called `mex`, it compile the single and multiple source MEX functions. Note that the `buildfile` was tested on Windows, with MATLAB R2025a. +Here is a `buildfile` with a single task called `mex`, this task compile the single and multiple source MEX functions and places them within the `private` folder. Note that the `buildfile` was tested on MATLAB R2025a. ``` matlab function plan = buildfile From 4be33150e0e3b0b1ca11c52476e1315247198bc8 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 11:48:57 +0530 Subject: [PATCH 027/105] Added notes on incorporating external libraries --- MEX.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index 63a5dab..8d94571 100644 --- a/MEX.md +++ b/MEX.md @@ -98,7 +98,7 @@ arithmetic/ └───buildfile.m ``` -Here is a `buildfile` with a single task called `mex`, this task compile the single and multiple source MEX functions and places them within the `private` folder. Note that the `buildfile` was tested on MATLAB R2025a. +Here is a `buildfile` with a single task called `mex`, this task compiles the single and multiple source MEX functions and places them within the `private` folder. Note that the `buildfile` was tested on MATLAB R2025a. ``` matlab function plan = buildfile @@ -119,6 +119,12 @@ end ## Incorporating External Libraries +One of the key strengths of MEX functions is their ability to call libraries implemented in languages like C, C++ and Fortran. Since MEX source files are just source code written in a MEX supported language, they use the source language syntax to access functionality implemented in other libraries. One of the common questions with using external libraries in you toolbox work is, where to store these external libraries? + +The answer to this question depends on if the library is required at **compile** or **run** time. For the purposes of this best practices we will assume that the external library is available to you as a include headers and binaries. We do not dwell into the details of how these binaries are created. + + + * When your MEX function relies on external libraries, store the binaries in a `libraries` directory with platform-specific subdirectories, as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. * For projects with complex dependencies, consider adopting dependency management tools like [Conan](https://conan.io/) which can significantly simplify library management across different platforms. * During the build process, copy the external libraries into the `toolbox/private` folder to circumvent path management issues on Windows and `LD_LIBRARY_PATH` concerns on Linux. From 2a806d1eeb4bafdb1e24f5781a1e0516457aee93 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 12:29:42 +0530 Subject: [PATCH 028/105] Added notes for managing external library headers --- MEX.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/MEX.md b/MEX.md index 8d94571..7d1753c 100644 --- a/MEX.md +++ b/MEX.md @@ -117,13 +117,23 @@ plan("mex") = MexTask.forEachFile(mexSourceFiles, mexOutputFolder); end ``` +### Call MEX functions from a mexhost + ## Incorporating External Libraries One of the key strengths of MEX functions is their ability to call libraries implemented in languages like C, C++ and Fortran. Since MEX source files are just source code written in a MEX supported language, they use the source language syntax to access functionality implemented in other libraries. One of the common questions with using external libraries in you toolbox work is, where to store these external libraries? The answer to this question depends on if the library is required at **compile** or **run** time. For the purposes of this best practices we will assume that the external library is available to you as a include headers and binaries. We do not dwell into the details of how these binaries are created. +### External library headers +These files are required only at compile time, your toolbox users do not want them to run the MEX functions. Having a standard location to store these headers makes it easier for you (the author) to manage them and pass it to the compiler. + +Create an `include` folder within the the language specific folder (say `cpp`) and move the external library headers within this folder. The [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) MATLAB API takes in an [optional argument](https://www.mathworks.com/help/matlab/ref/mex.html#btw17rw-1-option1optionN) that can be used to inform the compiler to look for header files within the `include` folder. + + +### Managing compile time library binaries +The toolbox user does not need these libraries to run the MEX functions that make a call to these libraries. * When your MEX function relies on external libraries, store the binaries in a `libraries` directory with platform-specific subdirectories, as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. * For projects with complex dependencies, consider adopting dependency management tools like [Conan](https://conan.io/) which can significantly simplify library management across different platforms. From 91720b62e681a9812c65f7427f1cc302f565ebdb Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 12:50:05 +0530 Subject: [PATCH 029/105] Added notes for compile time binaries --- MEX.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index 7d1753c..e3a63e5 100644 --- a/MEX.md +++ b/MEX.md @@ -133,7 +133,14 @@ Create an `include` folder within the the language specific folder (say `cpp`) a ### Managing compile time library binaries -The toolbox user does not need these libraries to run the MEX functions that make a call to these libraries. +The toolbox user does not need these libraries to run the MEX functions they are required only at compile time. You can place these binaries under platform specific folders within the `library` folder. We recommend using standard names for these platform folders. + +| Platform | Folder name | +| :---------------- | :------: | +| Linux | glnx64 | +| Windows | win64 | +| Mac ARM | maca64 | + * When your MEX function relies on external libraries, store the binaries in a `libraries` directory with platform-specific subdirectories, as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. * For projects with complex dependencies, consider adopting dependency management tools like [Conan](https://conan.io/) which can significantly simplify library management across different platforms. From 793687999c92dcfa21a9d7b5e27dc5de42dee2da Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 12:57:23 +0530 Subject: [PATCH 030/105] Added the table for file extensions and folder names --- MEX.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/MEX.md b/MEX.md index e3a63e5..6d39cfc 100644 --- a/MEX.md +++ b/MEX.md @@ -133,13 +133,13 @@ Create an `include` folder within the the language specific folder (say `cpp`) a ### Managing compile time library binaries -The toolbox user does not need these libraries to run the MEX functions they are required only at compile time. You can place these binaries under platform specific folders within the `library` folder. We recommend using standard names for these platform folders. +The toolbox user does not need these libraries to run the MEX functions they are required only at compile time, these libraries are often referred to as static libraries. You can place these binaries under platform specific folders within the `library` folder. We recommend using standard names for the platform folders. The table below provides a summary of the folder names and file extensions used for static libraries for popular platforms -| Platform | Folder name | -| :---------------- | :------: | -| Linux | glnx64 | -| Windows | win64 | -| Mac ARM | maca64 | +| Platform | Folder name | File Extension | +| :---------------- | :------ | :------ | +| Linux | glnx64 | .a | +| Windows | win64 | .lib | +| Mac ARM | maca64 | .dylib | * When your MEX function relies on external libraries, store the binaries in a `libraries` directory with platform-specific subdirectories, as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. From 7a0eb7d0222a3c2e4b53d82458a9f64d612251e4 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 13:06:23 +0530 Subject: [PATCH 031/105] Added notes for out of process mex functions --- MEX.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index 6d39cfc..2c1ee00 100644 --- a/MEX.md +++ b/MEX.md @@ -117,7 +117,8 @@ plan("mex") = MexTask.forEachFile(mexSourceFiles, mexOutputFolder); end ``` -### Call MEX functions from a mexhost +### Out of process MEX host +If you are using MEX functions written using the C++, we recommend calling the MEX function from an out of process MEX host. You can create an out of process MEX host using the [mexhost](https://www.mathworks.com/help/matlab/ref/mexhost.html) command. It protects MATLAB from crashing due to unexpected errors originating from the MEX function execution. ## Incorporating External Libraries @@ -142,6 +143,10 @@ The toolbox user does not need these libraries to run the MEX functions they ar | Mac ARM | maca64 | .dylib | +### Managing runtime library binaries + + + * When your MEX function relies on external libraries, store the binaries in a `libraries` directory with platform-specific subdirectories, as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. * For projects with complex dependencies, consider adopting dependency management tools like [Conan](https://conan.io/) which can significantly simplify library management across different platforms. * During the build process, copy the external libraries into the `toolbox/private` folder to circumvent path management issues on Windows and `LD_LIBRARY_PATH` concerns on Linux. From b9cd463f076c8d3b3e6597eebb3a6a75e0cfa021 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 14:47:12 +0530 Subject: [PATCH 032/105] added notes for runtime libraries --- MEX.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/MEX.md b/MEX.md index 2c1ee00..c987a7a 100644 --- a/MEX.md +++ b/MEX.md @@ -134,17 +134,17 @@ Create an `include` folder within the the language specific folder (say `cpp`) a ### Managing compile time library binaries -The toolbox user does not need these libraries to run the MEX functions they are required only at compile time, these libraries are often referred to as static libraries. You can place these binaries under platform specific folders within the `library` folder. We recommend using standard names for the platform folders. The table below provides a summary of the folder names and file extensions used for static libraries for popular platforms +The toolbox user does not need these libraries to run the MEX functions they are required only at compile time, these libraries are often referred to as static libraries. You can place these binaries under platform specific folders within the `library` folder. We recommend using standard names for the platform folders. The table below provides a summary of the folder names and file extensions used for static libraries for popular platforms. | Platform | Folder name | File Extension | | :---------------- | :------ | :------ | -| Linux | glnx64 | .a | +| Linux | glnxa64 | .a | | Windows | win64 | .lib | -| Mac ARM | maca64 | .dylib | - +| ARM Mac | maca64 | .dylib | +| Intel Mac | maci64 | .dylib | ### Managing runtime library binaries - +These type of libraries are often referred to as shared object libraries or static libraries. These libraries are required by the toolbox user to the run the MEX functions. You can place the runtime binaries within the `private` folder under the `toolbox` folder, this makes sure that the library gets shipped to the user. You can use the -L and -l flags during compile time to specify the location and the name of the runtime library. * When your MEX function relies on external libraries, store the binaries in a `libraries` directory with platform-specific subdirectories, as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. From 199286f1c7705967b6f46d5ccdb6b3f023c6add3 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Apr 2025 14:51:51 +0530 Subject: [PATCH 033/105] renamed runtime to execution time --- MEX.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/MEX.md b/MEX.md index c987a7a..62c88ef 100644 --- a/MEX.md +++ b/MEX.md @@ -124,7 +124,7 @@ If you are using MEX functions written using the C++, we recommend calling the M ## Incorporating External Libraries One of the key strengths of MEX functions is their ability to call libraries implemented in languages like C, C++ and Fortran. Since MEX source files are just source code written in a MEX supported language, they use the source language syntax to access functionality implemented in other libraries. One of the common questions with using external libraries in you toolbox work is, where to store these external libraries? -The answer to this question depends on if the library is required at **compile** or **run** time. For the purposes of this best practices we will assume that the external library is available to you as a include headers and binaries. We do not dwell into the details of how these binaries are created. +The answer to this question depends on if the library is required at **compile** or **execution** time. For the purposes of this best practices we will assume that the external library is available to you as a include headers and binaries. We do not dwell into the details of how these binaries are created. ### External library headers These files are required only at compile time, your toolbox users do not want them to run the MEX functions. Having a standard location to store these headers makes it easier for you (the author) to manage them and pass it to the compiler. @@ -133,8 +133,8 @@ Create an `include` folder within the the language specific folder (say `cpp`) a -### Managing compile time library binaries -The toolbox user does not need these libraries to run the MEX functions they are required only at compile time, these libraries are often referred to as static libraries. You can place these binaries under platform specific folders within the `library` folder. We recommend using standard names for the platform folders. The table below provides a summary of the folder names and file extensions used for static libraries for popular platforms. +### Compile time libraries +The toolbox user does not need these library binaries to run the MEX functions they are required only at compile time, these libraries are often referred to as static libraries. You can place these binaries under platform specific folders within the `library` folder. We recommend using standard names for the platform folders. The table below provides a summary of the folder names and file extensions used for static libraries for popular platforms. | Platform | Folder name | File Extension | | :---------------- | :------ | :------ | @@ -143,8 +143,8 @@ The toolbox user does not need these libraries to run the MEX functions they ar | ARM Mac | maca64 | .dylib | | Intel Mac | maci64 | .dylib | -### Managing runtime library binaries -These type of libraries are often referred to as shared object libraries or static libraries. These libraries are required by the toolbox user to the run the MEX functions. You can place the runtime binaries within the `private` folder under the `toolbox` folder, this makes sure that the library gets shipped to the user. You can use the -L and -l flags during compile time to specify the location and the name of the runtime library. +### Execution time libraries +These type of libraries are often referred to as shared object libraries or static libraries. These libraries are required for running the MEX functions and need to be shipped to the users. You can place the runtime binaries within the `private` folder under the `toolbox` folder, this makes sure that the library gets shipped to the user. You can use the -L and -l flags during compile time to specify the location and the name of the runtime library. * When your MEX function relies on external libraries, store the binaries in a `libraries` directory with platform-specific subdirectories, as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. From 15b708f3624c0ef5c2e27de67efacc61f9968b85 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 22 Apr 2025 07:58:48 +0530 Subject: [PATCH 034/105] Added the example folder structure for execution time and compile time external libraries --- MEX.md | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/MEX.md b/MEX.md index 62c88ef..f4501b2 100644 --- a/MEX.md +++ b/MEX.md @@ -143,10 +143,58 @@ The toolbox user does not need these library binaries to run the MEX functions | ARM Mac | maca64 | .dylib | | Intel Mac | maci64 | .dylib | +``` text +zlibStatic/ +: +├───cpp/ +| ├───include/ +| │ └───zlib.h +| ├───library/ +| │ ├───glnxa64 +| | | └───libz.a +| │ ├───win64 +| | | └───libz.lib +| │ └───maca64 +| | └───libz.dylib +│ └───mexfunctions/ +│ └───deflateMex.cpp +├───toolbox/ +| ├───deflate.m +| └───private/ +| ├───deflateMex.mexw64 (derived) +| ├───deflateMex.mexa64 (derived) +| └───deflateMex.mexmaca64 (derived) +├───zlibStatic.prj +└───buildfile.m +``` + + ### Execution time libraries These type of libraries are often referred to as shared object libraries or static libraries. These libraries are required for running the MEX functions and need to be shipped to the users. You can place the runtime binaries within the `private` folder under the `toolbox` folder, this makes sure that the library gets shipped to the user. You can use the -L and -l flags during compile time to specify the location and the name of the runtime library. + +``` text +zlibShared/ +: +├───cpp/ +| ├───include/ +| │ └───zlib.h +│ └───mexfunctions/ +│ └───deflateMex.cpp +├───toolbox/ +| ├───deflate.m +| └───private/ +| ├───libz.so +| ├───z.lib +| ├───libz.dylib +| ├───deflateMex.mexw64 (derived) +| ├───deflateMex.mexa64 (derived) +| └───deflateMex.mexmaca64 (derived) +├───zlibShared.prj +└───buildfile.m +``` + * When your MEX function relies on external libraries, store the binaries in a `libraries` directory with platform-specific subdirectories, as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. * For projects with complex dependencies, consider adopting dependency management tools like [Conan](https://conan.io/) which can significantly simplify library management across different platforms. * During the build process, copy the external libraries into the `toolbox/private` folder to circumvent path management issues on Windows and `LD_LIBRARY_PATH` concerns on Linux. From 0fc9ed87f10b8fca47a807f782c052d62c2fe26c Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 22 Apr 2025 09:52:57 +0530 Subject: [PATCH 035/105] minor update --- MEX.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/MEX.md b/MEX.md index f4501b2..fe554de 100644 --- a/MEX.md +++ b/MEX.md @@ -122,9 +122,9 @@ If you are using MEX functions written using the C++, we recommend calling the M ## Incorporating External Libraries -One of the key strengths of MEX functions is their ability to call libraries implemented in languages like C, C++ and Fortran. Since MEX source files are just source code written in a MEX supported language, they use the source language syntax to access functionality implemented in other libraries. One of the common questions with using external libraries in you toolbox work is, where to store these external libraries? +One of the key strengths of MEX functions is their ability to call libraries implemented in languages like C, C++ and Fortran. Since MEX source files are just source code written in a MEX supported language, they use the source language syntax to access functionality implemented in external libraries. One of the common questions with using external libraries in you toolbox work is, where to store these external libraries? -The answer to this question depends on if the library is required at **compile** or **execution** time. For the purposes of this best practices we will assume that the external library is available to you as a include headers and binaries. We do not dwell into the details of how these binaries are created. +The answer to this question depends on if the library is required at **compile** or **execution** time. For the purposes of this best practices, we will assume that the external library is available to you as include headers and binaries. We do not dwell into the details of how these binaries are created. ### External library headers These files are required only at compile time, your toolbox users do not want them to run the MEX functions. Having a standard location to store these headers makes it easier for you (the author) to manage them and pass it to the compiler. @@ -134,9 +134,9 @@ Create an `include` folder within the the language specific folder (say `cpp`) a ### Compile time libraries -The toolbox user does not need these library binaries to run the MEX functions they are required only at compile time, these libraries are often referred to as static libraries. You can place these binaries under platform specific folders within the `library` folder. We recommend using standard names for the platform folders. The table below provides a summary of the folder names and file extensions used for static libraries for popular platforms. +The toolbox user does not need these library binaries to run the MEX functions they are required only at compile time, these libraries are often referred to as static libraries. You can place these binaries under platform specific folders within the `library` folder. We recommend using standard names for the platform folders as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. The table below provides a summary of the folder names and file extensions used for static libraries for popular platforms. -| Platform | Folder name | File Extension | +| Platform | Folder name | Binary Extension | | :---------------- | :------ | :------ | | Linux | glnxa64 | .a | | Windows | win64 | .lib | @@ -173,7 +173,6 @@ zlibStatic/ These type of libraries are often referred to as shared object libraries or static libraries. These libraries are required for running the MEX functions and need to be shipped to the users. You can place the runtime binaries within the `private` folder under the `toolbox` folder, this makes sure that the library gets shipped to the user. You can use the -L and -l flags during compile time to specify the location and the name of the runtime library. - ``` text zlibShared/ : @@ -195,6 +194,7 @@ zlibShared/ └───buildfile.m ``` + * When your MEX function relies on external libraries, store the binaries in a `libraries` directory with platform-specific subdirectories, as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. * For projects with complex dependencies, consider adopting dependency management tools like [Conan](https://conan.io/) which can significantly simplify library management across different platforms. * During the build process, copy the external libraries into the `toolbox/private` folder to circumvent path management issues on Windows and `LD_LIBRARY_PATH` concerns on Linux. From ebd5a2b2690f4a271ca13986ecefc5f8fe206f0b Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 22 Apr 2025 10:02:41 +0530 Subject: [PATCH 036/105] Added notes for header only libraries --- MEX.md | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/MEX.md b/MEX.md index fe554de..e7f7516 100644 --- a/MEX.md +++ b/MEX.md @@ -124,13 +124,19 @@ If you are using MEX functions written using the C++, we recommend calling the M ## Incorporating External Libraries One of the key strengths of MEX functions is their ability to call libraries implemented in languages like C, C++ and Fortran. Since MEX source files are just source code written in a MEX supported language, they use the source language syntax to access functionality implemented in external libraries. One of the common questions with using external libraries in you toolbox work is, where to store these external libraries? -The answer to this question depends on if the library is required at **compile** or **execution** time. For the purposes of this best practices, we will assume that the external library is available to you as include headers and binaries. We do not dwell into the details of how these binaries are created. +The answer to this question depends on the external library. For the purposes of this best practices we classify the external libraries into, + 1. compile time libraries + 2. execution time time. + +We will assume that the external library is available to you as include headers and binaries. We do not dwell into the details of how these headers and binaries are created. ### External library headers These files are required only at compile time, your toolbox users do not want them to run the MEX functions. Having a standard location to store these headers makes it easier for you (the author) to manage them and pass it to the compiler. Create an `include` folder within the the language specific folder (say `cpp`) and move the external library headers within this folder. The [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) MATLAB API takes in an [optional argument](https://www.mathworks.com/help/matlab/ref/mex.html#btw17rw-1-option1optionN) that can be used to inform the compiler to look for header files within the `include` folder. +If you want to include a header only library in your work, copy the library headers into the `include` folder within the respective language folders. + ### Compile time libraries @@ -161,9 +167,9 @@ zlibStatic/ ├───toolbox/ | ├───deflate.m | └───private/ -| ├───deflateMex.mexw64 (derived) -| ├───deflateMex.mexa64 (derived) -| └───deflateMex.mexmaca64 (derived) +| ├───deflateMex.mexw64 +| ├───deflateMex.mexa64 +| └───deflateMex.mexmaca64 ├───zlibStatic.prj └───buildfile.m ``` @@ -185,11 +191,11 @@ zlibShared/ | ├───deflate.m | └───private/ | ├───libz.so -| ├───z.lib +| ├───z.dll | ├───libz.dylib -| ├───deflateMex.mexw64 (derived) -| ├───deflateMex.mexa64 (derived) -| └───deflateMex.mexmaca64 (derived) +| ├───deflateMex.mexw64 +| ├───deflateMex.mexa64 +| └───deflateMex.mexmaca64 ├───zlibShared.prj └───buildfile.m ``` From 42a42bc10bbd2c1264107bb888fb4b4dd19dd135 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 22 Apr 2025 10:09:47 +0530 Subject: [PATCH 037/105] Added notes for test --- MEX.md | 28 ++++++++++------------------ 1 file changed, 10 insertions(+), 18 deletions(-) diff --git a/MEX.md b/MEX.md index e7f7516..04d1428 100644 --- a/MEX.md +++ b/MEX.md @@ -240,13 +240,22 @@ arithmetic/ ## Testing -Develop tests for the MATLAB layer in the `tests/` folder. While MEX functionality is indirectly tested through MATLAB scripts, this approach ensures comprehensive validation of your toolbox's capabilities. +Develop tests for the MATLAB layer in the `tests` folder. While MEX functionality is indirectly tested through MATLAB scripts, this approach ensures comprehensive validation of your toolbox's capabilities. Our example toolbox adds `testAdd.m` and `testSubtract.m`to validate the functionality of `add.m` and `subtract.m` which, in turn, use `addMex` and `subtractMex`. ``` text arithmetic/ : +├───tests/ +| ├───testAdd.m +| └───testSubstract.m +├───cpp/ +│ ├───substractMex/ +│ │ ├───substract.cpp +│ │ └───substractImp.cpp +│ └───mexfunctions/ +│ └───addMex.cpp ├───toolbox/ | ├───add.m | ├───subtract.m @@ -257,23 +266,6 @@ arithmetic/ | ├───subtractMex.mexw64 (derived) | ├───subtractMex.mexa64 (derived) | └───subtractMex.mexmaca64 (derived) -├───cpp/ -│ ├───addMex/ -│ │ ├───firstFile.cpp -│ │ └───secondFile.cpp -│ └───subtractMex/ -│ ├───subtract.cpp -| └───libraries -| ├───complex.hpp -| ├───glnxa64 -| | └───libcomplex.so -| ├───maci64 -| | └───libcomplex.dylib -| └───win64 -| └───complex.dll -├───tests/ -| ├───testAdd.m -| └───testSubstract.m ├───arithmetic.prj └───buildfile.m ``` From f8e1bb0c19ea5a01b942953de176f789299214a0 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 24 Apr 2025 14:50:16 +0530 Subject: [PATCH 038/105] Minor updates --- MEX.md | 26 ++++++++++++-------------- 1 file changed, 12 insertions(+), 14 deletions(-) diff --git a/MEX.md b/MEX.md index 04d1428..ffa2bd5 100644 --- a/MEX.md +++ b/MEX.md @@ -19,7 +19,7 @@ To illustrate these best practices, we've created a sample project: The Arithmet - **MEX Gateway Function**: Written in a MEX supported language, MATLAB accesses this function when a call is made to the MEX function. Each MEX function can have one gateway function only. [MEX gateway functions written in C](https://www.mathworks.com/help/matlab/apiref/mexfunction.html) - **Compile time binaries**: Static libraries are a good examples of compile time binaries. These library binaries are required only at build time and you need not ship them to your users. - **Run time binaries**: These are platform dependent binaries, that the users need to run your toolbox, shared object libraries (.so files) in Linux and dynamic link libraries (.dll files) in Windows are good examples. -- **`buildtool`**: MATLAB's tool for automating build processes, including MEX file compilation. See the [`buildtool` documentation](https://www.mathworks.com/help/matlab/ref/buildtool.html) for more information. +- **`buildtool`**: MATLAB's tool for automating build processes, including MEX file compilation. See the [`buildtool`](https://www.mathworks.com/help/matlab/matlab_prog/overview-of-matlab-build-tool.html) for more information. - **MEX compiler**: Converts MEX source files into MEX functions. The MEX compiler can be accessed from MATLAB via the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, it can be invoked for the system terminal via the same command. You can configure C/ C++/ Fortran compilers using the [mex -setup](https://www.mathworks.com/help/matlab/matlab_external/changing-default-compiler.html). - **Project Root**: The folder under which the toolbox source code is organized. A git repository is initialized under this folder. - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. @@ -39,7 +39,7 @@ For C and C++ single source MEX functions, place the source code within the `mex For multiple source MEX functions, create a folder for each MEX function within the respective language folder. The name of the folder should be same as that of the MEX function that will be created out for source code with the folder. The folder name can be suffixed with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function, with the same name as the folder. -Our [Arithmetic Toolbox]() example features two MEX functions: `addMex` and `subtractMex`. `addMex` is implemented as a single source MEX function and the source code is placed under the `mexfunctions` folder. On the other hand, `substractMex` is implemented as a multiple source MEX function with source code placed under the `substractMex` folder, `substract.cpp` implements the MEX gateway function for the `substractMex` MEX function and `substractImp.cpp` implementations functionalities that are used in `substract.cpp`. The organization of the MEX source files is shown below: +Our [Arithmetic Toolbox]() example features two MEX functions: `addMex` and `subtractMex`. `addMex` is implemented as a single source MEX function with the source code placed under the `mexfunctions` folder. On the other hand, `substractMex` is implemented as a multiple source MEX function with source code placed under the `substractMex` folder, `substract.cpp` implements the MEX gateway function for the `substractMex` MEX function and `substractImp.cpp` implementations functionalities that are used in `substract.cpp`. The organization of the MEX source files is shown below: ``` text arithmetic/ @@ -55,6 +55,7 @@ arithmetic/ ├───arithmetic.prj └───buildfile.m ``` + ## Building MEX Functions - **Tools**: Use MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b, to automate the MEX build process. The [`matlab.buildtool.tasks.MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html), introduced in R2024a, automates the compilation, ensuring consistency across environments. If you need support in earlier releases, use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command directly. @@ -124,11 +125,9 @@ If you are using MEX functions written using the C++, we recommend calling the M ## Incorporating External Libraries One of the key strengths of MEX functions is their ability to call libraries implemented in languages like C, C++ and Fortran. Since MEX source files are just source code written in a MEX supported language, they use the source language syntax to access functionality implemented in external libraries. One of the common questions with using external libraries in you toolbox work is, where to store these external libraries? -The answer to this question depends on the external library. For the purposes of this best practices we classify the external libraries into, - 1. compile time libraries - 2. execution time time. +The answer to this question depends on the external library and the platform with which you are working with. For the purposes of this best practices we classify the external libraries into compile time and execution time libraries. -We will assume that the external library is available to you as include headers and binaries. We do not dwell into the details of how these headers and binaries are created. +We will assume that the external library is available to you as include headers and binaries. We do not dwell into the details of how these headers and binaries are created. First let as talk a bit about organizing ### External library headers These files are required only at compile time, your toolbox users do not want them to run the MEX functions. Having a standard location to store these headers makes it easier for you (the author) to manage them and pass it to the compiler. @@ -154,6 +153,7 @@ zlibStatic/ : ├───cpp/ | ├───include/ +| │ ├───zlibconf.h | │ └───zlib.h | ├───library/ | │ ├───glnxa64 @@ -176,7 +176,7 @@ zlibStatic/ ### Execution time libraries -These type of libraries are often referred to as shared object libraries or static libraries. These libraries are required for running the MEX functions and need to be shipped to the users. You can place the runtime binaries within the `private` folder under the `toolbox` folder, this makes sure that the library gets shipped to the user. You can use the -L and -l flags during compile time to specify the location and the name of the runtime library. +These type of libraries are often referred to as shared object libraries or dynamic link libraries. These libraries are required for running the MEX functions and need to be shipped to the users. You can place the runtime binaries within the `private` folder under the `toolbox` folder, this makes sure that the library gets shipped to the user. You can use the -L and -l flags during compile time to specify the location and the name of the runtime library. ``` text @@ -191,7 +191,7 @@ zlibShared/ | ├───deflate.m | └───private/ | ├───libz.so -| ├───z.dll +| ├───libz.lib | ├───libz.dylib | ├───deflateMex.mexw64 | ├───deflateMex.mexa64 @@ -199,15 +199,13 @@ zlibShared/ ├───zlibShared.prj └───buildfile.m ``` - - -* When your MEX function relies on external libraries, store the binaries in a `libraries` directory with platform-specific subdirectories, as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. + * For projects with complex dependencies, consider adopting dependency management tools like [Conan](https://conan.io/) which can significantly simplify library management across different platforms. * During the build process, copy the external libraries into the `toolbox/private` folder to circumvent path management issues on Windows and `LD_LIBRARY_PATH` concerns on Linux. -Our example toolbox adds a library called `complex` to the `subtractMex` function: + -``` text + ## Testing From 84089246119d3c2dcbb52b1a3999f7b2ea2afd66 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 24 Apr 2025 15:07:09 +0530 Subject: [PATCH 039/105] Added notes on loader search path --- MEX.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/MEX.md b/MEX.md index ffa2bd5..99d2586 100644 --- a/MEX.md +++ b/MEX.md @@ -176,7 +176,7 @@ zlibStatic/ ### Execution time libraries -These type of libraries are often referred to as shared object libraries or dynamic link libraries. These libraries are required for running the MEX functions and need to be shipped to the users. You can place the runtime binaries within the `private` folder under the `toolbox` folder, this makes sure that the library gets shipped to the user. You can use the -L and -l flags during compile time to specify the location and the name of the runtime library. +These type of libraries are often referred to as shared object libraries or dynamic link libraries. These libraries are required for running the MEX functions and need to be shipped to the users. You can place the execution time binaries within the `private` folder under the `toolbox` folder, this makes sure that the library gets shipped to the user. You can use the -L and -l flags during compile time to specify the location and the name of the runtime library. ``` text @@ -201,7 +201,16 @@ zlibShared/ ``` * For projects with complex dependencies, consider adopting dependency management tools like [Conan](https://conan.io/) which can significantly simplify library management across different platforms. -* During the build process, copy the external libraries into the `toolbox/private` folder to circumvent path management issues on Windows and `LD_LIBRARY_PATH` concerns on Linux. +* Depending on the platform, execution time libraries require their path to be added to the loader's search path. These search paths are often establish using platform dependent environment variables. In case of C++ libraries you can use [`mexhost`](https://www.mathworks.com/help/matlab/ref/mexhost.html)'s `EnvironmentVariables` option to set-up the loader's path. Here is a summary of +loader's search path for different platforms. + +| Platform | Environment variable for loader's search path | +| :-------- | :-------------------------------------------- | +| Linux | LD_LIBRARY_PATH | +| Windows | Path | +| Mac | LD_LIBRARY_PATH | + +For non C++ libraries, you need to start MATLAB from an environment where the linker's search path is already established. @@ -281,4 +290,4 @@ We welcome your input! For further details, suggestions, or to contribute, pleas --- [![CC-BY-4.0](images/cc-by-40.png)](https://creativecommons.org/licenses/by/4.0/) -Copyright © 2023-2024, The MathWorks, Inc. +Copyright © 2023-2025, The MathWorks, Inc. From 499e969e05d903b3b1cf4524aacc6d5083afc25a Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 5 May 2025 12:34:06 +0530 Subject: [PATCH 040/105] Updated the notes for buildfile --- MEX.md | 45 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 40 insertions(+), 5 deletions(-) diff --git a/MEX.md b/MEX.md index 99d2586..0089ed8 100644 --- a/MEX.md +++ b/MEX.md @@ -24,6 +24,42 @@ To illustrate these best practices, we've created a sample project: The Arithmet - **Project Root**: The folder under which the toolbox source code is organized. A git repository is initialized under this folder. - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. +## Scenario: 1 Single source MEX functions +Suppose that you have a bunch of MEX source files that + +``` text +arithmetic/ +├───cpp/ +│ └───mexfunctions/ +| ├───substractMex.cpp +│ └───addMex.cpp +├───toolbox/ +| ├───add.m +| └───subtract.m +├───arithmetic.prj +└───buildfile.m +``` + +### Building the MEX functions +MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b can be used to automate building the MEX functions. Here is a simple buildfile for building MEX functions: +``` matlab +function plan = buildfile + +% Create a plan +plan = buildplan(); + +% Compile all the .cpp files inside cpp/mexfunctions into MEX functions +mexSourceFiles = files(plan, fullfile("cpp", "mexfunctions", "*.cpp")); +mexOutputFolder = fullfile("toolbox","private"); +plan("mex") = matlab.buildtool.tasks.MexTask.forEachFile(mexSourceFiles, mexOutputFolder); + +end +``` +In the buildfile, we create a [plan](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [MexTask](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the plan. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html), introduced in R2025a, converts every C++ file within the folder into MEX functions. [MexTask.forEachFile](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [FileCollection](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input. A FileCollecton object can also be created using the [files](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API + +automates the compilation, ensuring consistency across environments. If you need support in earlier releases, use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command directly. + + ## Organizing MEX Source Files Organize your MEX source files in language-specific folder at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. We recommend using the `cpp` folder at the root of the toolbox repo for both C and C++ source code. @@ -56,8 +92,7 @@ arithmetic/ └───buildfile.m ``` -## Building MEX Functions -- **Tools**: Use MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b, to automate the MEX build process. The [`matlab.buildtool.tasks.MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html), introduced in R2024a, automates the compilation, ensuring consistency across environments. If you need support in earlier releases, use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command directly. + ## Organizing MEX Functions @@ -207,10 +242,10 @@ loader's search path for different platforms. | Platform | Environment variable for loader's search path | | :-------- | :-------------------------------------------- | | Linux | LD_LIBRARY_PATH | -| Windows | Path | -| Mac | LD_LIBRARY_PATH | +| Windows | PATH | +| Mac | DYLD_LIBRARY_PATH | -For non C++ libraries, you need to start MATLAB from an environment where the linker's search path is already established. +For non C++ libraries, you need to start MATLAB from an environment where the loaders's search path is already established. From 6290847313de7be9d776744abd06fd008a693fd7 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 5 May 2025 12:43:45 +0530 Subject: [PATCH 041/105] Completed the build section --- MEX.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/MEX.md b/MEX.md index 0089ed8..f1273b0 100644 --- a/MEX.md +++ b/MEX.md @@ -55,9 +55,11 @@ plan("mex") = matlab.buildtool.tasks.MexTask.forEachFile(mexSourceFiles, mexOutp end ``` -In the buildfile, we create a [plan](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [MexTask](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the plan. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html), introduced in R2025a, converts every C++ file within the folder into MEX functions. [MexTask.forEachFile](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [FileCollection](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input. A FileCollecton object can also be created using the [files](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API +In the buildfile, we create a [`plan`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the plan. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html), introduced in R2025a, converts every C++ file within the folder into MEX functions. [`MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [`FileCollection`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input. A FileCollecton object can also be created using the [`files`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API. The buildfile has some hidden benefits, +1. Incremental build +2. No hardcoding of source files -automates the compilation, ensuring consistency across environments. If you need support in earlier releases, use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command directly. +Since we are using `MexTask` for building MEX files directly as opposed to using the `mex` command, we get support for incremental build out of box. Moreover, since we are pointing only to the folder that contain the MEX source files, the buildfile can be extended for any number of MEX source files as long as the source files are placed within the `cpp/mexfunctions` folder ## Organizing MEX Source Files From dc1d8930ec7c397d12cd7b5eba598488fd2d3d8c Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 6 May 2025 12:04:56 +0530 Subject: [PATCH 042/105] Adding notes for scenario 1 --- MEX.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/MEX.md b/MEX.md index f1273b0..f7902db 100644 --- a/MEX.md +++ b/MEX.md @@ -25,7 +25,11 @@ To illustrate these best practices, we've created a sample project: The Arithmet - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. ## Scenario: 1 Single source MEX functions -Suppose that you have a bunch of MEX source files that +Suppose you have several C/C++ MEX source files, each implementing its own MEX gateway and none of these files depend on an external library or other source files. Since the C/C++ source files need not be distributed to the toolbox users, we recommend placing these source files outside the `toolbox` folder. Create folder named `cpp` under the toolbox repository root. Within this folder, create a subfolder called `mexfunctions` and place the source code for all the single source MEX functions within the `mexfunctions` subfolder. + +We will be using the arithmetic repository, for demonstrating the idea. For the sack of discussion, we will reimplement the functionality of `add()` and `substruct()` using MEX functions. We have created two MEX source files `addMex.cpp` and `substractMex.cpp`. The names of the C/C++ source files reflect the names of the MEX functions complied out of these source files. On Windows platform `addMex.cpp` would be complied to `addMex.mexw64`. Having the same file name (excluding extension)makes it easy to go between the MEX functions and their source code. + +Notice that we have suffixed the filenames of the source code with "Mex", to ind ``` text arithmetic/ @@ -41,7 +45,7 @@ arithmetic/ ``` ### Building the MEX functions -MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b can be used to automate building the MEX functions. Here is a simple buildfile for building MEX functions: +MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b can be used to automate building the MEX functions. Here is a simple buildfile for can be applied on the previously discussed folder structure: ``` matlab function plan = buildfile From 3126cdbc32d47f7f2b612572a61ae52ecb47295e Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 6 May 2025 14:42:43 +0530 Subject: [PATCH 043/105] Minor updates to the wording --- MEX.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/MEX.md b/MEX.md index f7902db..8362cbc 100644 --- a/MEX.md +++ b/MEX.md @@ -25,11 +25,11 @@ To illustrate these best practices, we've created a sample project: The Arithmet - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. ## Scenario: 1 Single source MEX functions -Suppose you have several C/C++ MEX source files, each implementing its own MEX gateway and none of these files depend on an external library or other source files. Since the C/C++ source files need not be distributed to the toolbox users, we recommend placing these source files outside the `toolbox` folder. Create folder named `cpp` under the toolbox repository root. Within this folder, create a subfolder called `mexfunctions` and place the source code for all the single source MEX functions within the `mexfunctions` subfolder. +Suppose you have several C/C++ MEX source files, each implementing its own MEX gateway and not depending on an external library or other source files. Since the C/C++ source files need not be distributed to the toolbox users, we recommend placing them outside the `toolbox` folder. Create a folder named `cpp` under the repository root. Within this folder, create a subfolder called `mexfunctions` and place the single source MEX functions source code within this folder. -We will be using the arithmetic repository, for demonstrating the idea. For the sack of discussion, we will reimplement the functionality of `add()` and `substruct()` using MEX functions. We have created two MEX source files `addMex.cpp` and `substractMex.cpp`. The names of the C/C++ source files reflect the names of the MEX functions complied out of these source files. On Windows platform `addMex.cpp` would be complied to `addMex.mexw64`. Having the same file name (excluding extension)makes it easy to go between the MEX functions and their source code. +To illustrate this organization, we will use the arithmetic repository as an example. For the demonstration purposes we will reimplement the internals of `add()` and `substruct()` using MEX functions. We have created two MEX source files: `addMex.cpp` and `substractMex.cpp`. The names of the C/C++ source files are chosen to match the the MEX functions complied out of the source files. For example, on Windows platform, `addMex.cpp` would be complied to `addMex.mexw64` and `addMex` will be the name of the MEX function . Maintaining consistent file names (apart from the extension) makes it easy to associate each MEX function with its source code. Moreover, the "Mex" suffix in the file name clearly indicates that the function is a MEX function rather than a standard MATLAB function. -Notice that we have suffixed the filenames of the source code with "Mex", to ind +The organization of the MEX source files relative to the toolbox folder is shown below: ``` text arithmetic/ @@ -60,6 +60,7 @@ plan("mex") = matlab.buildtool.tasks.MexTask.forEachFile(mexSourceFiles, mexOutp end ``` In the buildfile, we create a [`plan`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the plan. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html), introduced in R2025a, converts every C++ file within the folder into MEX functions. [`MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [`FileCollection`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input. A FileCollecton object can also be created using the [`files`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API. The buildfile has some hidden benefits, + 1. Incremental build 2. No hardcoding of source files From 778c7dfb1595f110e903b40ca5e6ced443b11167 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 6 May 2025 15:02:51 +0530 Subject: [PATCH 044/105] refactored the advantages of buildfile --- MEX.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/MEX.md b/MEX.md index 8362cbc..f9e6898 100644 --- a/MEX.md +++ b/MEX.md @@ -45,7 +45,7 @@ arithmetic/ ``` ### Building the MEX functions -MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b can be used to automate building the MEX functions. Here is a simple buildfile for can be applied on the previously discussed folder structure: +We recommend using MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b for automating the MEX functions build. Here is a simple buildfile that can be used to build single source MEX functions: ``` matlab function plan = buildfile @@ -59,13 +59,13 @@ plan("mex") = matlab.buildtool.tasks.MexTask.forEachFile(mexSourceFiles, mexOutp end ``` -In the buildfile, we create a [`plan`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the plan. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html), introduced in R2025a, converts every C++ file within the folder into MEX functions. [`MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [`FileCollection`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input. A FileCollecton object can also be created using the [`files`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API. The buildfile has some hidden benefits, +In the buildfile, we create a [`plan`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the it. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) API, introduced in R2025a, converts every C++ file within the specified folder into MEX functions. [`MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [`FileCollection`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input, which can be created using the [`files`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API. -1. Incremental build -2. No hardcoding of source files - -Since we are using `MexTask` for building MEX files directly as opposed to using the `mex` command, we get support for incremental build out of box. Moreover, since we are pointing only to the folder that contain the MEX source files, the buildfile can be extended for any number of MEX source files as long as the source files are placed within the `cpp/mexfunctions` folder +The buildfile has some advantages, +1. *Incremental build:* By using `MexTask` instead of the `mex` command, the build process automatically supports incremental build, recompiling only the files that have changed. +2. *No hardcoding of source files:* Since the buildfile references the folder containing the MEX source files, there is no need to manually specify or update the list of source files. +3. *Platform independence:* Although MEX functions themselves are platform dependent, the buildfile does not rely on any platform-specific information to compile them, making it possible to use the same buildfile across different platforms. ## Organizing MEX Source Files Organize your MEX source files in language-specific folder at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. We recommend using the `cpp` folder at the root of the toolbox repo for both C and C++ source code. From 9995b9c77f9928fd0f689b1bb649f2f671259be4 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 6 May 2025 15:28:21 +0530 Subject: [PATCH 045/105] minor beautification --- MEX.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/MEX.md b/MEX.md index f9e6898..18f2ce9 100644 --- a/MEX.md +++ b/MEX.md @@ -67,6 +67,8 @@ The buildfile has some advantages, 2. *No hardcoding of source files:* Since the buildfile references the folder containing the MEX source files, there is no need to manually specify or update the list of source files. 3. *Platform independence:* Although MEX functions themselves are platform dependent, the buildfile does not rely on any platform-specific information to compile them, making it possible to use the same buildfile across different platforms. +With this setup, the build file can easily scale to handle any number of single-source MEX functions, as long as the source files are placed within the cpp/mexfunctions folder. + ## Organizing MEX Source Files Organize your MEX source files in language-specific folder at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. We recommend using the `cpp` folder at the root of the toolbox repo for both C and C++ source code. From 9fa6bc8457e13947e95cb50406c5eb6b22cc7a66 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 6 May 2025 21:52:37 +0530 Subject: [PATCH 046/105] Added the organization of files after build --- MEX.md | 34 +++++++++++++++++++++++++++++++--- 1 file changed, 31 insertions(+), 3 deletions(-) diff --git a/MEX.md b/MEX.md index 18f2ce9..7ad5ffc 100644 --- a/MEX.md +++ b/MEX.md @@ -24,7 +24,7 @@ To illustrate these best practices, we've created a sample project: The Arithmet - **Project Root**: The folder under which the toolbox source code is organized. A git repository is initialized under this folder. - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. -## Scenario: 1 Single source MEX functions +## Organizing single source MEX functions Suppose you have several C/C++ MEX source files, each implementing its own MEX gateway and not depending on an external library or other source files. Since the C/C++ source files need not be distributed to the toolbox users, we recommend placing them outside the `toolbox` folder. Create a folder named `cpp` under the repository root. Within this folder, create a subfolder called `mexfunctions` and place the single source MEX functions source code within this folder. To illustrate this organization, we will use the arithmetic repository as an example. For the demonstration purposes we will reimplement the internals of `add()` and `substruct()` using MEX functions. We have created two MEX source files: `addMex.cpp` and `substractMex.cpp`. The names of the C/C++ source files are chosen to match the the MEX functions complied out of the source files. For example, on Windows platform, `addMex.cpp` would be complied to `addMex.mexw64` and `addMex` will be the name of the MEX function . Maintaining consistent file names (apart from the extension) makes it easy to associate each MEX function with its source code. Moreover, the "Mex" suffix in the file name clearly indicates that the function is a MEX function rather than a standard MATLAB function. @@ -44,8 +44,36 @@ arithmetic/ └───buildfile.m ``` -### Building the MEX functions -We recommend using MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b for automating the MEX functions build. Here is a simple buildfile that can be used to build single source MEX functions: +### Building MEX functions +We recommend using MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b for automating MEX functions build. + +[MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder, contents of this folder gets shipped to the user. For a toolbox that uses MEX, place the MEX functions under a `private` folder within the `toolbox` folder, thus making them private [Private functions](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). + +Our motivation for making MEX functions private is to restrict the toolbox user from calling them directly. We recommend accessing the MEX functions always from a MATLAB script, this approach gives toolbox authors control over what gets passed as input to the MEX functions, their by elimination failures due to unhandled inputs. + +We recommend ignoring the MEX functions from version control (add the `private` folder to the `.gitignore`) since MEX functions are derived artifacts. + +``` text +arithmetic/ +├───cpp/ +│ └───mexfunctions/ +| ├───substractMex.cpp +│ └───addMex.cpp +├───toolbox/ +| ├───private/ +| | ├───substractMex.mexw64 +| | ├───substractMex.mexw64 +| | ├───substractMex.mexw64 +| | ├───addMex.mexw64 +| | ├───addMex.mexw64 +│ | └───addMex.mexw64 +| ├───add.m +| └───subtract.m +├───arithmetic.prj +└───buildfile.m +``` + +Here is a simple buildfile that can be used to build single source MEX functions, these MEX functions will be stored within private folder within the toolbox folder: ``` matlab function plan = buildfile From 4cae121d408580a155c2bf12bf159c494ee34050 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Fri, 9 May 2025 16:39:53 +0530 Subject: [PATCH 047/105] Updated the language --- MEX.md | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/MEX.md b/MEX.md index 7ad5ffc..729b283 100644 --- a/MEX.md +++ b/MEX.md @@ -25,11 +25,11 @@ To illustrate these best practices, we've created a sample project: The Arithmet - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. ## Organizing single source MEX functions -Suppose you have several C/C++ MEX source files, each implementing its own MEX gateway and not depending on an external library or other source files. Since the C/C++ source files need not be distributed to the toolbox users, we recommend placing them outside the `toolbox` folder. Create a folder named `cpp` under the repository root. Within this folder, create a subfolder called `mexfunctions` and place the single source MEX functions source code within this folder. +Suppose you have several C++ MEX source files, each implementing its own MEX gateway and not depending on an external library or other source files, you want to organize these files in a way that makes building, integrating the sharing the MEX functions easy. Since the C++ source files need not be distributed to the toolbox users, we recommend placing them outside the `toolbox` folder. Create a folder named `cpp` under the repository root. Within this folder, create a subfolder called `mexfunctions` and place the single source MEX functions source code within this folder. -To illustrate this organization, we will use the arithmetic repository as an example. For the demonstration purposes we will reimplement the internals of `add()` and `substruct()` using MEX functions. We have created two MEX source files: `addMex.cpp` and `substractMex.cpp`. The names of the C/C++ source files are chosen to match the the MEX functions complied out of the source files. For example, on Windows platform, `addMex.cpp` would be complied to `addMex.mexw64` and `addMex` will be the name of the MEX function . Maintaining consistent file names (apart from the extension) makes it easy to associate each MEX function with its source code. Moreover, the "Mex" suffix in the file name clearly indicates that the function is a MEX function rather than a standard MATLAB function. +To illustrate this organization, we will use the arithmetic repository as an example. For the demonstration purposes we will reimplement the internals of `add()` and `substruct()` using MEX functions. We have created two MEX source files: `addMex.cpp` and `substractMex.cpp`. The names of the C/C++ source files are chosen to match the the MEX functions complied out of the source files. For example, on Windows platform, `addMex.cpp` would be complied to `addMex.mexw64` and `addMex` will be the name of the MEX function . Maintaining consistent file names (apart from the extension) makes it easy to associate each MEX function with its source code. Moreover, the "Mex" suffix in the MEX function file name clearly indicates that the function is a MEX function rather than a standard MATLAB function. -The organization of the MEX source files relative to the toolbox folder is shown below: +The organization of the MEX source files for the arithmetic toolbox repository relative to the toolbox folder is shown below: ``` text arithmetic/ @@ -47,7 +47,7 @@ arithmetic/ ### Building MEX functions We recommend using MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b for automating MEX functions build. -[MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder, contents of this folder gets shipped to the user. For a toolbox that uses MEX, place the MEX functions under a `private` folder within the `toolbox` folder, thus making them private [Private functions](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). +[MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder, contents of this folder gets shipped to the user. Create a `private` folder within the `toolbox` folder and place the MEX functions within this folder, thus making them private [Private functions](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). Our motivation for making MEX functions private is to restrict the toolbox user from calling them directly. We recommend accessing the MEX functions always from a MATLAB script, this approach gives toolbox authors control over what gets passed as input to the MEX functions, their by elimination failures due to unhandled inputs. @@ -73,7 +73,7 @@ arithmetic/ └───buildfile.m ``` -Here is a simple buildfile that can be used to build single source MEX functions, these MEX functions will be stored within private folder within the toolbox folder: +Here is a simple buildfile that can be used to build single source MEX functions: ``` matlab function plan = buildfile @@ -87,15 +87,16 @@ plan("mex") = matlab.buildtool.tasks.MexTask.forEachFile(mexSourceFiles, mexOutp end ``` -In the buildfile, we create a [`plan`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the it. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) API, introduced in R2025a, converts every C++ file within the specified folder into MEX functions. [`MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [`FileCollection`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input, which can be created using the [`files`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API. +In the build file, we create a [`plan`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the it. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) API, introduced in R2025a, converts every C++ file within the specified folder into MEX functions. [`MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [`FileCollection`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input, which can be created using the [`files`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API. The buildfile has some advantages, -1. *Incremental build:* By using `MexTask` instead of the `mex` command, the build process automatically supports incremental build, recompiling only the files that have changed. -2. *No hardcoding of source files:* Since the buildfile references the folder containing the MEX source files, there is no need to manually specify or update the list of source files. -3. *Platform independence:* Although MEX functions themselves are platform dependent, the buildfile does not rely on any platform-specific information to compile them, making it possible to use the same buildfile across different platforms. +1. *Incremental build:* By using `MexTask` instead of the `mex` command, you automatically get support for incremental build, recompiling only the files that have changed. +2. *No hardcoding of source files:* Since the build file references to the folder containing the MEX source files and not the source files directly, the build file can scale to handle any number of single-source MEX functions, as long as the source files are placed within the cpp/mexfunctions folder. +3. *Platform independence:* Although MEX functions themselves are platform dependent, the build file does not rely on any platform-specific information to compile them, making it possible to use the same build file across different platforms. + +### Multi platform MED functions build using CI systems -With this setup, the build file can easily scale to handle any number of single-source MEX functions, as long as the source files are placed within the cpp/mexfunctions folder. ## Organizing MEX Source Files Organize your MEX source files in language-specific folder at the root level of your project (e.g., `cpp`). This structure enhances code organization and simplifies management across multiple programming languages. We recommend using the `cpp` folder at the root of the toolbox repo for both C and C++ source code. From eaaff1ea7853c53d4d4555e6815af9af6a5c0667 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Fri, 9 May 2025 16:56:43 +0530 Subject: [PATCH 048/105] minor update --- MEX.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/MEX.md b/MEX.md index 729b283..1b9c5ad 100644 --- a/MEX.md +++ b/MEX.md @@ -5,11 +5,11 @@ ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) -Welcome to the MATLAB® MEX Best Practice guide. This document offers a refined approach to seamlessly integrating MEX files into your MATLAB toolbox projects, complementing our established [MATLAB Toolbox Best Practices](README.md). MEX files enable you to harness the power of C /C++/ Fortran functions within MATLAB, and this guide will equip you with the knowledge to maintain a consistent and efficient workflow. Let's explore the world of MEX integration! +Welcome to the MATLAB® MEX Best Practice guide. This document offers a refined approach to seamlessly integrating MEX files into your MATLAB toolbox projects, complementing our established [MATLAB Toolbox Best Practices](README.md). MEX files enable you to harness the power of C/C++/Fortran functions within MATLAB, and this guide will equip you with the knowledge to maintain a consistent and efficient workflow. Let's explore the world of MEX integration! ## Overview -MEX functions bridge the gap between MATLAB and C, C++, or Fortran, allowing you to leverage the strengths of these languages directly within MATLAB. While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This organized structure enhances maintainability and makes it easier for others to understand and contribute to your project. +[MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) bridge the gap between MATLAB and C, C++, or Fortran, allowing you to leverage the strengths of these languages directly within MATLAB. While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This organized structure enhances maintainability and makes it easier for others to understand and contribute to your project. To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. @@ -25,11 +25,11 @@ To illustrate these best practices, we've created a sample project: The Arithmet - **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. ## Organizing single source MEX functions -Suppose you have several C++ MEX source files, each implementing its own MEX gateway and not depending on an external library or other source files, you want to organize these files in a way that makes building, integrating the sharing the MEX functions easy. Since the C++ source files need not be distributed to the toolbox users, we recommend placing them outside the `toolbox` folder. Create a folder named `cpp` under the repository root. Within this folder, create a subfolder called `mexfunctions` and place the single source MEX functions source code within this folder. +Suppose you have several C++ MEX source files, each implementing its own MEX gateway and not depending on an external library or other source files, you want to organize these files in a way that makes building, integrating and sharing the MEX functions easy. Since the C++ source files need not be distributed to the toolbox users, we recommend placing them outside the `toolbox` folder. Create a folder named `cpp` under the repository root. Within this folder, create a subfolder called `mexfunctions` and place the single source MEX functions source code within this folder. -To illustrate this organization, we will use the arithmetic repository as an example. For the demonstration purposes we will reimplement the internals of `add()` and `substruct()` using MEX functions. We have created two MEX source files: `addMex.cpp` and `substractMex.cpp`. The names of the C/C++ source files are chosen to match the the MEX functions complied out of the source files. For example, on Windows platform, `addMex.cpp` would be complied to `addMex.mexw64` and `addMex` will be the name of the MEX function . Maintaining consistent file names (apart from the extension) makes it easy to associate each MEX function with its source code. Moreover, the "Mex" suffix in the MEX function file name clearly indicates that the function is a MEX function rather than a standard MATLAB function. +To illustrate this organization, we will use the arithmetic repository as an example. For demonstration purposes we will reimplement the internals of `add()` and `substruct()` using MEX functions. We have created two MEX source files: `addMex.cpp` and `substractMex.cpp`. The names of the C/C++ source files are chosen to match the the MEX functions complied out of the source files. For example, on Windows platform, `addMex.cpp` would be complied to `addMex.mexw64` and `addMex` will be the name of the MEX function . Maintaining consistent file names (apart from the extension) makes it easy to associate each MEX function with its source code. Moreover, the "Mex" suffix in the MEX function file name clearly indicates that the function is a MEX function rather than a standard MATLAB function. -The organization of the MEX source files for the arithmetic toolbox repository relative to the toolbox folder is shown below: +The organization of the MEX source files for the arithmetic toolbox relative to the toolbox folder is shown below: ``` text arithmetic/ @@ -47,11 +47,11 @@ arithmetic/ ### Building MEX functions We recommend using MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b for automating MEX functions build. -[MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder, contents of this folder gets shipped to the user. Create a `private` folder within the `toolbox` folder and place the MEX functions within this folder, thus making them private [Private functions](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). +[MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder, contents of this folder gets shipped to the user. Create a `private` folder within the `toolbox` folder and place the MEX functions within this folder, thus making them [Private functions](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). Our motivation for making MEX functions private is to restrict the toolbox user from calling them directly. We recommend accessing the MEX functions always from a MATLAB script, this approach gives toolbox authors control over what gets passed as input to the MEX functions, their by elimination failures due to unhandled inputs. -We recommend ignoring the MEX functions from version control (add the `private` folder to the `.gitignore`) since MEX functions are derived artifacts. +We recommend ignoring the MEX functions from version control (add the `private` folder to the `.gitignore`) since MEX functions are derived artifacts. Here is the organization of the toolbox repository after building the MEX functions. ``` text arithmetic/ @@ -354,6 +354,9 @@ arithmetic/ ## Automating Builds with GitHub Actions **TBD** +## Summary: Organizing a toolbox with MEX functions + + ## Conclusion By following to these best practices, you'll create a robust, maintainable, and user-friendly MATLAB toolbox that harnesses the full potential of MEX files. Through effective project structure organization, build automation, and dependency management, you can focus on delivering great solutions to your users. From 4afd1b993ebb5901532a4142934ce2595ea6641b Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Fri, 9 May 2025 17:12:38 +0530 Subject: [PATCH 049/105] Added the first part for multiple source mex fuctions --- MEX.md | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/MEX.md b/MEX.md index 1b9c5ad..3151d17 100644 --- a/MEX.md +++ b/MEX.md @@ -49,7 +49,7 @@ We recommend using MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ [MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder, contents of this folder gets shipped to the user. Create a `private` folder within the `toolbox` folder and place the MEX functions within this folder, thus making them [Private functions](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). -Our motivation for making MEX functions private is to restrict the toolbox user from calling them directly. We recommend accessing the MEX functions always from a MATLAB script, this approach gives toolbox authors control over what gets passed as input to the MEX functions, their by elimination failures due to unhandled inputs. +Our motivation for making MEX functions private is to restrict the toolbox user from calling them directly. We recommend accessing MEX functions always from a MATLAB script, this approach gives toolbox authors control over what gets passed as input to the MEX functions, their by elimination failures due to unhandled inputs. We recommend ignoring the MEX functions from version control (add the `private` folder to the `.gitignore`) since MEX functions are derived artifacts. Here is the organization of the toolbox repository after building the MEX functions. @@ -95,7 +95,29 @@ The buildfile has some advantages, 2. *No hardcoding of source files:* Since the build file references to the folder containing the MEX source files and not the source files directly, the build file can scale to handle any number of single-source MEX functions, as long as the source files are placed within the cpp/mexfunctions folder. 3. *Platform independence:* Although MEX functions themselves are platform dependent, the build file does not rely on any platform-specific information to compile them, making it possible to use the same build file across different platforms. -### Multi platform MED functions build using CI systems +### Multi platform MEX functions build using CI systems + + +## Organizing multiple source MEX functions +It is common to have MEX functions generated from multiple C++ source files, we call these MEX functions multiple source MEX functions. For such MEX functions one of the source files implement the gateway function and other source files implement methods that are called within the gateway function. For multiple source MEX functions, create a folder for each MEX function within the `cpp` folder. The name of the folder should be same as that of the MEX function that will be created out for source code with the folder. The folder name can be suffixed with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. + + +For the sake of discussion let us implement substractMex MEXfunction as a Multiple source MEX function. The organization of the source files is shown below. + +``` text +arithmetic/ +├───cpp/ +│ ├───substractMex/ +│ │ ├───substract.cpp % Implements gateway function +│ │ └───substractImp.cpp % other features +│ └───mexfunctions/ +│ └───addMex.cpp +├───toolbox/ +| ├───add.m +| └───subtract.m +├───arithmetic.prj +└───buildfile.m +``` ## Organizing MEX Source Files From f8b6f25a9015758d053331ee0ae7ea5e14ef1e6e Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Fri, 9 May 2025 22:09:09 +0530 Subject: [PATCH 050/105] Updated after working meeting with Rob Co-authored-by: Rob Purser --- MEX.md | 64 +++++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 43 insertions(+), 21 deletions(-) diff --git a/MEX.md b/MEX.md index 3151d17..62a8318 100644 --- a/MEX.md +++ b/MEX.md @@ -5,32 +5,48 @@ ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) -Welcome to the MATLAB® MEX Best Practice guide. This document offers a refined approach to seamlessly integrating MEX files into your MATLAB toolbox projects, complementing our established [MATLAB Toolbox Best Practices](README.md). MEX files enable you to harness the power of C/C++/Fortran functions within MATLAB, and this guide will equip you with the knowledge to maintain a consistent and efficient workflow. Let's explore the world of MEX integration! + + +_This is a sub document of the Toolbox Best practices._ +Welcome to the MATLAB® MEX Best Practice guide. This document offers a refined approach to seamlessly integrating MEX files into your MATLAB toolbox projects, complementing our established [MATLAB Toolbox Best Practices](README.md). MEX files enable you to harness the power of C/C++/Fortran functions within MATLAB, and this guide will equip you with the knowledge to maintain a consistent and efficient workflow. _We're going show you how to...._ ## Overview -[MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) bridge the gap between MATLAB and C, C++, or Fortran, allowing you to leverage the strengths of these languages directly within MATLAB. While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This organized structure enhances maintainability and makes it easier for others to understand and contribute to your project. +[MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) bridge the gap between MATLAB and C, C++, or Fortran, allowing you to leverage the strengths of these languages directly within MATLAB. While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This makes it easier for others to understand and contribute to your project. + +To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. To make things clearer, when we say "C++", we mean "C, C++, and Fortran." -To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. ## Key Concepts -- **MEX Source Files**: These are functions written in C, C++, or Fortran, compiled to be callable from MATLAB. See the [MEX documentation](https://www.mathworks.com/help/matlab/cpp-mex-file-applications.html) for more information. -- **MEX Functions**: Platform dependent binaries that can be called directly from MATLAB, they behave almost like a MATLAB function. MATLAB uses a platform dependent file extension for MEX functions, you can determine the MEX extension for your platform using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). -- **MEX Gateway Function**: Written in a MEX supported language, MATLAB accesses this function when a call is made to the MEX function. Each MEX function can have one gateway function only. [MEX gateway functions written in C](https://www.mathworks.com/help/matlab/apiref/mexfunction.html) -- **Compile time binaries**: Static libraries are a good examples of compile time binaries. These library binaries are required only at build time and you need not ship them to your users. -- **Run time binaries**: These are platform dependent binaries, that the users need to run your toolbox, shared object libraries (.so files) in Linux and dynamic link libraries (.dll files) in Windows are good examples. -- **`buildtool`**: MATLAB's tool for automating build processes, including MEX file compilation. See the [`buildtool`](https://www.mathworks.com/help/matlab/matlab_prog/overview-of-matlab-build-tool.html) for more information. -- **MEX compiler**: Converts MEX source files into MEX functions. The MEX compiler can be accessed from MATLAB via the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, it can be invoked for the system terminal via the same command. You can configure C/ C++/ Fortran compilers using the [mex -setup](https://www.mathworks.com/help/matlab/matlab_external/changing-default-compiler.html). -- **Project Root**: The folder under which the toolbox source code is organized. A git repository is initialized under this folder. -- **CI/CD Pipelines**: Continuous Integration and Continuous Deployment tools like GitHub Actions or GitLab CI/CD ensure your code is tested and deployed automatically. - -## Organizing single source MEX functions + +_See [Toolbox Best Practices] for other concepts._ +- **MEX Functions**: Compiled code that can be called directly from MATLAB. They behave like a MATLAB function. MEX files are platform dependent binaries, so there is a platform dependent file extension. You can determine the MEX extension for your platform using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). + + + + + + + + + + +## Creating a MEX function from a single C++ source file + + Suppose you have several C++ MEX source files, each implementing its own MEX gateway and not depending on an external library or other source files, you want to organize these files in a way that makes building, integrating and sharing the MEX functions easy. Since the C++ source files need not be distributed to the toolbox users, we recommend placing them outside the `toolbox` folder. Create a folder named `cpp` under the repository root. Within this folder, create a subfolder called `mexfunctions` and place the single source MEX functions source code within this folder. -To illustrate this organization, we will use the arithmetic repository as an example. For demonstration purposes we will reimplement the internals of `add()` and `substruct()` using MEX functions. We have created two MEX source files: `addMex.cpp` and `substractMex.cpp`. The names of the C/C++ source files are chosen to match the the MEX functions complied out of the source files. For example, on Windows platform, `addMex.cpp` would be complied to `addMex.mexw64` and `addMex` will be the name of the MEX function . Maintaining consistent file names (apart from the extension) makes it easy to associate each MEX function with its source code. Moreover, the "Mex" suffix in the MEX function file name clearly indicates that the function is a MEX function rather than a standard MATLAB function. + +To illustrate this organization, we will use the arithmetic repository as an example. For demonstration purposes we will reimplement the internals of `add()` and `substruct()` using MEX functions. We have created two MEX source files: `addMex.cpp` and `substractMex.cpp`. The names of the C/C++ source files are chosen to match the the MEX functions complied out of the source files. For example, on Windows platform, `addMex.cpp` would be complied to `addMex.mexw64` and `addMex` will be the name of the MEX function . +This naming convention gives us +* makes it easy to associate each MEX function with its source code. +* the "Mex" suffix in the MEX function file name clearly indicates that the function is a MEX function rather than a standard MATLAB function. + +The organization of the MEX source files for the arithmetic toolbox relative to the toolbox folder is shown below: + ``` text arithmetic/ ├───cpp/ @@ -45,6 +61,7 @@ arithmetic/ ``` ### Building MEX functions + We recommend using MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b for automating MEX functions build. [MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder, contents of this folder gets shipped to the user. Create a `private` folder within the `toolbox` folder and place the MEX functions within this folder, thus making them [Private functions](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). @@ -62,17 +79,18 @@ arithmetic/ ├───toolbox/ | ├───private/ | | ├───substractMex.mexw64 -| | ├───substractMex.mexw64 -| | ├───substractMex.mexw64 -| | ├───addMex.mexw64 -| | ├───addMex.mexw64 │ | └───addMex.mexw64 | ├───add.m | └───subtract.m ├───arithmetic.prj └───buildfile.m ``` + +## Expanding to create multiple MEX functions +_Insert the add and subtract example here_ +### Automating using `buildtool` + Here is a simple buildfile that can be used to build single source MEX functions: ``` matlab function plan = buildfile @@ -87,8 +105,11 @@ plan("mex") = matlab.buildtool.tasks.MexTask.forEachFile(mexSourceFiles, mexOutp end ``` + + In the build file, we create a [`plan`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the it. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) API, introduced in R2025a, converts every C++ file within the specified folder into MEX functions. [`MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [`FileCollection`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input, which can be created using the [`files`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API. + The buildfile has some advantages, 1. *Incremental build:* By using `MexTask` instead of the `mex` command, you automatically get support for incremental build, recompiling only the files that have changed. @@ -98,7 +119,8 @@ The buildfile has some advantages, ### Multi platform MEX functions build using CI systems -## Organizing multiple source MEX functions +## Creating a MEX function from multiple C++ source files + It is common to have MEX functions generated from multiple C++ source files, we call these MEX functions multiple source MEX functions. For such MEX functions one of the source files implement the gateway function and other source files implement methods that are called within the gateway function. For multiple source MEX functions, create a folder for each MEX function within the `cpp` folder. The name of the folder should be same as that of the MEX function that will be created out for source code with the folder. The folder name can be suffixed with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. From d0429787e002999303ba90e19b24af76ba21deee Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 12 May 2025 11:06:32 +0530 Subject: [PATCH 051/105] Added the definition of MEX functions within the overview section. --- MEX.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/MEX.md b/MEX.md index 62a8318..7e7ccac 100644 --- a/MEX.md +++ b/MEX.md @@ -12,15 +12,15 @@ Welcome to the MATLAB® MEX Best Practice guide. This document offers a refin ## Overview -[MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) bridge the gap between MATLAB and C, C++, or Fortran, allowing you to leverage the strengths of these languages directly within MATLAB. While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This makes it easier for others to understand and contribute to your project. +[MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) are compiler code that bridges the gap between MATLAB and C, C++, or Fortran. They behave like a MATLAB function and are platform dependent. You can determine the MEX extension for your platform using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This makes it easier for others to understand and contribute to your project. -To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. To make things clearer, when we say "C++", we mean "C, C++, and Fortran." +To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. To make things clearer, when we say "C++", we mean "C, C++, and Fortran." For Key Concepts please refer to [Toolbox Best Practices](./README.md). - -## Key Concepts - + _See [Toolbox Best Practices] for other concepts._ -- **MEX Functions**: Compiled code that can be called directly from MATLAB. They behave like a MATLAB function. MEX files are platform dependent binaries, so there is a platform dependent file extension. You can determine the MEX extension for your platform using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). + + From 0694bb47370f08a72420db0bb78f419f711f00b0 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 15 May 2025 11:16:26 +0530 Subject: [PATCH 052/105] updated the single source mex file use case --- MEX.md | 42 +++++++++++++++++++++++++++--------------- 1 file changed, 27 insertions(+), 15 deletions(-) diff --git a/MEX.md b/MEX.md index 7e7ccac..a79bc08 100644 --- a/MEX.md +++ b/MEX.md @@ -5,20 +5,24 @@ ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) - + -_This is a sub document of the Toolbox Best practices._ -Welcome to the MATLAB® MEX Best Practice guide. This document offers a refined approach to seamlessly integrating MEX files into your MATLAB toolbox projects, complementing our established [MATLAB Toolbox Best Practices](README.md). MEX files enable you to harness the power of C/C++/Fortran functions within MATLAB, and this guide will equip you with the knowledge to maintain a consistent and efficient workflow. _We're going show you how to...._ + + +Welcome to the MATLAB® MEX Best Practice guide, this document extends [MATLAB Toolbox Best Practices](./README.md) by offering advise on integrating [MEX files](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) into your MATLAB toolbox projects. MEX function enable you to harness the power of _C/C++/Fortran_ functions within MATLAB, and this guide will help you with organizing the MEX source code files and MEX functions. + + ## Overview [MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) are compiler code that bridges the gap between MATLAB and C, C++, or Fortran. They behave like a MATLAB function and are platform dependent. You can determine the MEX extension for your platform using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This makes it easier for others to understand and contribute to your project. -To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. To make things clearer, when we say "C++", we mean "C, C++, and Fortran." For Key Concepts please refer to [Toolbox Best Practices](./README.md). +To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. To make things clearer, when we say "C++", we mean "C, C++, and Fortran." For Key Concepts please refer to [Toolbox Best Practices](./README.md). -_See [Toolbox Best Practices] for other concepts._ + BP: Moved MEX functions to Overview + RP: See [Toolbox Best Practices] for other concepts--> @@ -34,16 +38,18 @@ _See [Toolbox Best Practices] for other concepts._ ## Creating a MEX function from a single C++ source file -Suppose you have several C++ MEX source files, each implementing its own MEX gateway and not depending on an external library or other source files, you want to organize these files in a way that makes building, integrating and sharing the MEX functions easy. Since the C++ source files need not be distributed to the toolbox users, we recommend placing them outside the `toolbox` folder. Create a folder named `cpp` under the repository root. Within this folder, create a subfolder called `mexfunctions` and place the single source MEX functions source code within this folder. +Suppose you have a C++ MEX source file. This file has got its own MEX gateway and doesn't depend on any external library or other source files. You want to organize this file in a way that makes building, integrating and sharing the MEX function easy. According the [Toolbox best practices](./README.md), MEX (C++) source files need not be distributed to the toolbox users, since they are not required to run the toolbox. So, here’s what we suggest: keep your MEX source file outside of the `toolbox` folder. You can do this by creating a folder called `cpp` at the `root` folder. Inside the `cpp` folder, make another folder named `mexfunctions`, and that’s where you’ll put your MEX function’s source code. -To illustrate this organization, we will use the arithmetic repository as an example. For demonstration purposes we will reimplement the internals of `add()` and `substruct()` using MEX functions. We have created two MEX source files: `addMex.cpp` and `substractMex.cpp`. The names of the C/C++ source files are chosen to match the the MEX functions complied out of the source files. For example, on Windows platform, `addMex.cpp` would be complied to `addMex.mexw64` and `addMex` will be the name of the MEX function . -This naming convention gives us -* makes it easy to associate each MEX function with its source code. -* the "Mex" suffix in the MEX function file name clearly indicates that the function is a MEX function rather than a standard MATLAB function. +Let’s walk through how this organization works with the arithmetic toolbox example. We implement a MATLAB function called `invertNumber()`, under the hood, this function uses the MEX functions `invertMex()`, which we compiled from `invertMex.cpp`. You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, and `invertMex` will be the name of the MEX function. + +This naming convention is really helpful for a couple of reasons: + +* First, it makes it easy to match each MEX function to its source code. +* Second, by adding the "Mex" suffix, it’s immediately clear that this is a MEX function—not a regular MATLAB function. The organization of the MEX source files for the arithmetic toolbox relative to the toolbox folder is shown below: @@ -51,18 +57,24 @@ The organization of the MEX source files for the arithmetic toolbox relative to arithmetic/ ├───cpp/ │ └───mexfunctions/ -| ├───substractMex.cpp -│ └───addMex.cpp +│ └───invertMex.cpp ├───toolbox/ | ├───add.m -| └───subtract.m +| ├───subtract.m +| └───invertNumber.m ├───arithmetic.prj └───buildfile.m ``` ### Building MEX functions -We recommend using MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b for automating MEX functions build. +The following [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, when executed from the `root folder` can builds the MEX function and place it within a `private` folder under the `toolbox` folder. +```matlab +>>source = fullfile("cpp", "mexfunctions", "invertMex.cpp"); +>>destinations = fullfile("toolbox", "private"); +>>mex(source, destination) +``` +When you have multiple MEX files, executing the `mex` command separately for each file MEX file can be cumbersome. Moreover, you might also becomes document your build recipe and share it with others. MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b is a solution to problems just mentioned. It provides tooling to automate your MEX build. [MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder, contents of this folder gets shipped to the user. Create a `private` folder within the `toolbox` folder and place the MEX functions within this folder, thus making them [Private functions](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). From b6ffd6207890e9ff334552a2dc60037ad310e19f Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 15 May 2025 12:03:27 +0530 Subject: [PATCH 053/105] Minor updates to build mex --- MEX.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/MEX.md b/MEX.md index a79bc08..91a38de 100644 --- a/MEX.md +++ b/MEX.md @@ -44,15 +44,16 @@ Suppose you have a C++ MEX source file. This file has got its own MEX gateway an Be able to say "We are going to create the invertMex function, which on Windows will be invertMex.mexw64." --> -Let’s walk through how this organization works with the arithmetic toolbox example. We implement a MATLAB function called `invertNumber()`, under the hood, this function uses the MEX functions `invertMex()`, which we compiled from `invertMex.cpp`. You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, and `invertMex` will be the name of the MEX function. +Let’s walk through how this organization works with the arithmetic toolbox example. We implement a MATLAB function `invertNumber()`, under the hood, this function uses the MEX functions `invertMex()`, which we compiled from `invertMex.cpp`. You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, and `invertMex` will be the name of the MEX function. -This naming convention is really helpful for a couple of reasons: +This naming convention is helpful for a couple of reasons: * First, it makes it easy to match each MEX function to its source code. * Second, by adding the "Mex" suffix, it’s immediately clear that this is a MEX function—not a regular MATLAB function. - -The organization of the MEX source files for the arithmetic toolbox relative to the toolbox folder is shown below: - + + ``` text arithmetic/ ├───cpp/ @@ -68,10 +69,10 @@ arithmetic/ ### Building MEX functions -The following [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, when executed from the `root folder` can builds the MEX function and place it within a `private` folder under the `toolbox` folder. +You can use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, to compile your MEX source code into MEX files. You need to provide the path to your MEX source file and where you want the compiled MEX file to go as inputs to the `mex` command. So, if you run the `mex` command from the `root` folder, you’d point to `cpp/mexfunctions/invertMex.cpp` as your source, and set `toolbox/private/` as the destination. ```matlab >>source = fullfile("cpp", "mexfunctions", "invertMex.cpp"); ->>destinations = fullfile("toolbox", "private"); +>>destination = fullfile("toolbox", "private"); >>mex(source, destination) ``` When you have multiple MEX files, executing the `mex` command separately for each file MEX file can be cumbersome. Moreover, you might also becomes document your build recipe and share it with others. MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b is a solution to problems just mentioned. It provides tooling to automate your MEX build. From 5b9f9ed0d7d973fd7b01f488c7834da991b81904 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 15 May 2025 14:48:37 +0530 Subject: [PATCH 054/105] Minor updates --- MEX.md | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/MEX.md b/MEX.md index 91a38de..286ce61 100644 --- a/MEX.md +++ b/MEX.md @@ -38,13 +38,13 @@ To illustrate these best practices, we've created a sample project: The Arithmet ## Creating a MEX function from a single C++ source file -Suppose you have a C++ MEX source file. This file has got its own MEX gateway and doesn't depend on any external library or other source files. You want to organize this file in a way that makes building, integrating and sharing the MEX function easy. According the [Toolbox best practices](./README.md), MEX (C++) source files need not be distributed to the toolbox users, since they are not required to run the toolbox. So, here’s what we suggest: keep your MEX source file outside of the `toolbox` folder. You can do this by creating a folder called `cpp` at the `root` folder. Inside the `cpp` folder, make another folder named `mexfunctions`, and that’s where you’ll put your MEX function’s source code. +Suppose you have a C++ MEX source file. This file has got its own MEX gateway and doesn't depend on any external library or other source files. You want to organize this file in a way that makes building, integrating and sharing the MEX function easy. According to the [Toolbox best practices](./README.md), MEX (C++) source files need not be distributed to the toolbox users, since they are not required to run the toolbox. So, here’s what we suggest: keep your MEX source file outside of the `toolbox` folder. You can do this by creating a folder called `cpp` at the `root` folder. Inside the `cpp` folder, make another folder named `mexfunctions`, and that’s where you put your MEX function’s source code. -Let’s walk through how this organization works with the arithmetic toolbox example. We implement a MATLAB function `invertNumber()`, under the hood, this function uses the MEX functions `invertMex()`, which we compiled from `invertMex.cpp`. You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, and `invertMex` will be the name of the MEX function. +Let’s walk through how this organization works with the arithmetic toolbox example. We implement a MATLAB function `invertNumber()`, under the hood, this function uses the MEX functions `invertMex()`, which we compiled from `invertMex.cpp`. You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, with `invertMex` being the name of the MEX function. This naming convention is helpful for a couple of reasons: @@ -75,32 +75,26 @@ You can use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) comm >>destination = fullfile("toolbox", "private"); >>mex(source, destination) ``` -When you have multiple MEX files, executing the `mex` command separately for each file MEX file can be cumbersome. Moreover, you might also becomes document your build recipe and share it with others. MATLAB's [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b is a solution to problems just mentioned. It provides tooling to automate your MEX build. -[MATLAB Toolbox Best Practices](README.md) advocates for placing the user files under the `toolbox` folder, contents of this folder gets shipped to the user. Create a `private` folder within the `toolbox` folder and place the MEX functions within this folder, thus making them [Private functions](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). - -Our motivation for making MEX functions private is to restrict the toolbox user from calling them directly. We recommend accessing MEX functions always from a MATLAB script, this approach gives toolbox authors control over what gets passed as input to the MEX functions, their by elimination failures due to unhandled inputs. - -We recommend ignoring the MEX functions from version control (add the `private` folder to the `.gitignore`) since MEX functions are derived artifacts. Here is the organization of the toolbox repository after building the MEX functions. +By placing the MEX file in a `private` folder, we have made the MEX function [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). Our motivation for doing this is to restrict the toolbox user from calling these functions directly. Instead, we suggest always accessing MEX functions through a MATLAB script. This way, you have control over what gets passed as input to the MEX function, this helps avoid errors from unexpected or unhandled inputs. +Also, since MEX files are just built artifacts, it’s a good idea to leave them out of version control. You can do this by adding the `private` folder to your `.gitignore` file—there’s no need to track those files in your repo. ``` text arithmetic/ ├───cpp/ │ └───mexfunctions/ -| ├───substractMex.cpp -│ └───addMex.cpp +│ └───invertMex.cpp ├───toolbox/ -| ├───private/ -| | ├───substractMex.mexw64 -│ | └───addMex.mexw64 +| ├───private +| | └───invertMex.mexw64 | ├───add.m -| └───subtract.m +| ├───subtract.m +| └───invertNumber.m ├───arithmetic.prj └───buildfile.m ``` -## Expanding to create multiple MEX functions -_Insert the add and subtract example here_ +If you’ve got several MEX functions, running the `mex` command for each one can be tedious. In addition, you might also want to document the build recipe, so that you can refer to it in the future or shared with others. That’s where MATLAB’s [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b can be useful. With buildtool, you can automate the MEX build process and share your build recipe with others. ### Automating using `buildtool` @@ -118,7 +112,7 @@ plan("mex") = matlab.buildtool.tasks.MexTask.forEachFile(mexSourceFiles, mexOutp end ``` - + In the build file, we create a [`plan`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the it. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) API, introduced in R2025a, converts every C++ file within the specified folder into MEX functions. [`MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [`FileCollection`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input, which can be created using the [`files`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API. @@ -129,6 +123,12 @@ The buildfile has some advantages, 2. *No hardcoding of source files:* Since the build file references to the folder containing the MEX source files and not the source files directly, the build file can scale to handle any number of single-source MEX functions, as long as the source files are placed within the cpp/mexfunctions folder. 3. *Platform independence:* Although MEX functions themselves are platform dependent, the build file does not rely on any platform-specific information to compile them, making it possible to use the same build file across different platforms. + +## Expanding to create multiple MEX functions +_Insert the add and subtract example here_ + + + ### Multi platform MEX functions build using CI systems From 42df472bd08fb2da1fe45f9b22f4bae4cd0eca46 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 15 May 2025 14:58:00 +0530 Subject: [PATCH 055/105] moved the automation using buildfile to the top --- MEX.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/MEX.md b/MEX.md index 286ce61..c9255a9 100644 --- a/MEX.md +++ b/MEX.md @@ -78,7 +78,7 @@ You can use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) comm By placing the MEX file in a `private` folder, we have made the MEX function [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). Our motivation for doing this is to restrict the toolbox user from calling these functions directly. Instead, we suggest always accessing MEX functions through a MATLAB script. This way, you have control over what gets passed as input to the MEX function, this helps avoid errors from unexpected or unhandled inputs. -Also, since MEX files are just built artifacts, it’s a good idea to leave them out of version control. You can do this by adding the `private` folder to your `.gitignore` file—there’s no need to track those files in your repo. +Also, since MEX files are just build artifacts, it’s a good idea to leave them out of version control. You can do this by adding the `private` folder to your `.gitignore` file—there’s no need to track those files in your repo. ``` text arithmetic/ ├───cpp/ @@ -94,11 +94,18 @@ arithmetic/ └───buildfile.m ``` -If you’ve got several MEX functions, running the `mex` command for each one can be tedious. In addition, you might also want to document the build recipe, so that you can refer to it in the future or shared with others. That’s where MATLAB’s [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b can be useful. With buildtool, you can automate the MEX build process and share your build recipe with others. - ### Automating using `buildtool` - -Here is a simple buildfile that can be used to build single source MEX functions: + +If you’ve got several MEX functions, running the `mex` command for each one can be tedious. In addition, you might also want to document the build recipe, so that you can refer to it in the future or shared with others. That’s where MATLAB’s [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b can be useful. Buildtool uses a build file to document the build recipe, which has several advantages: + + +1. *Incremental build:* By using `MexTask` instead of the `mex` command, you automatically get support for incremental build, recompiling only the files that have changed. +2. *No hardcoding of source files:* Since the build file references to the folder containing the MEX source files and not the source files directly, the build file can scale to handle any number of single-source MEX functions, as long as the source files are placed within the cpp/mexfunctions folder. +3. *Platform independence:* Although MEX functions themselves are platform dependent, the build file does not rely on any platform-specific information to compile them, making it possible to use the same build file across different platforms. + + ``` matlab function plan = buildfile @@ -116,12 +123,7 @@ end In the build file, we create a [`plan`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the it. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) API, introduced in R2025a, converts every C++ file within the specified folder into MEX functions. [`MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [`FileCollection`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input, which can be created using the [`files`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API. - -The buildfile has some advantages, -1. *Incremental build:* By using `MexTask` instead of the `mex` command, you automatically get support for incremental build, recompiling only the files that have changed. -2. *No hardcoding of source files:* Since the build file references to the folder containing the MEX source files and not the source files directly, the build file can scale to handle any number of single-source MEX functions, as long as the source files are placed within the cpp/mexfunctions folder. -3. *Platform independence:* Although MEX functions themselves are platform dependent, the build file does not rely on any platform-specific information to compile them, making it possible to use the same build file across different platforms. ## Expanding to create multiple MEX functions From 823fcc5d1a1021feffe3957a287f7a43f1d8fe63 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 15 May 2025 15:05:02 +0530 Subject: [PATCH 056/105] minor update --- MEX.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index c9255a9..ddb9234 100644 --- a/MEX.md +++ b/MEX.md @@ -101,7 +101,7 @@ If you’ve got several MEX functions, running the `mex` command for each one ca -1. *Incremental build:* By using `MexTask` instead of the `mex` command, you automatically get support for incremental build, recompiling only the files that have changed. +1. *Incremental build:* By using `MexTask` instead of the `mex` command, you automatically get support for incremental build, i.e) buildtool only re-compiles the files that have changed. 2. *No hardcoding of source files:* Since the build file references to the folder containing the MEX source files and not the source files directly, the build file can scale to handle any number of single-source MEX functions, as long as the source files are placed within the cpp/mexfunctions folder. 3. *Platform independence:* Although MEX functions themselves are platform dependent, the build file does not rely on any platform-specific information to compile them, making it possible to use the same build file across different platforms. From 067bf963b2c02c3ec9e155edf8473d55a1bea0db Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 15 May 2025 15:15:22 +0530 Subject: [PATCH 057/105] Responded to most of the feedback --- MEX.md | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/MEX.md b/MEX.md index ddb9234..0ffba2a 100644 --- a/MEX.md +++ b/MEX.md @@ -18,17 +18,19 @@ Welcome to the MATLAB® MEX Best Practice guide, this document extends [MATLA [MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) are compiler code that bridges the gap between MATLAB and C, C++, or Fortran. They behave like a MATLAB function and are platform dependent. You can determine the MEX extension for your platform using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This makes it easier for others to understand and contribute to your project. -To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. To make things clearer, when we say "C++", we mean "C, C++, and Fortran." For Key Concepts please refer to [Toolbox Best Practices](./README.md). +To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. To make things clearer, when we say "C++", we mean "C, C++, and Fortran." For key concepts please refer to the [Toolbox Best Practices](./README.md). + RP: See [Toolbox Best Practices] for other concepts + BP: Added a sentence in the previous paragraph--> - + @@ -37,12 +39,18 @@ To illustrate these best practices, we've created a sample project: The Arithmet ## Creating a MEX function from a single C++ source file - + Suppose you have a C++ MEX source file. This file has got its own MEX gateway and doesn't depend on any external library or other source files. You want to organize this file in a way that makes building, integrating and sharing the MEX function easy. According to the [Toolbox best practices](./README.md), MEX (C++) source files need not be distributed to the toolbox users, since they are not required to run the toolbox. So, here’s what we suggest: keep your MEX source file outside of the `toolbox` folder. You can do this by creating a folder called `cpp` at the `root` folder. Inside the `cpp` folder, make another folder named `mexfunctions`, and that’s where you put your MEX function’s source code. +BP: Introduced a new MATLAB function call invertNumbers and invertMex.cpp. + +RP: Be able to say "We are going to create the invertMex function, which on Windows will be invertMex.mexw64." + +BP: Added a statement similar to the one suggested--> Let’s walk through how this organization works with the arithmetic toolbox example. We implement a MATLAB function `invertNumber()`, under the hood, this function uses the MEX functions `invertMex()`, which we compiled from `invertMex.cpp`. You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, with `invertMex` being the name of the MEX function. @@ -68,7 +76,8 @@ arithmetic/ ``` ### Building MEX functions - + You can use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, to compile your MEX source code into MEX files. You need to provide the path to your MEX source file and where you want the compiled MEX file to go as inputs to the `mex` command. So, if you run the `mex` command from the `root` folder, you’d point to `cpp/mexfunctions/invertMex.cpp` as your source, and set `toolbox/private/` as the destination. ```matlab >>source = fullfile("cpp", "mexfunctions", "invertMex.cpp"); @@ -93,7 +102,8 @@ arithmetic/ ├───arithmetic.prj └───buildfile.m ``` - + ### Automating using `buildtool` @@ -119,11 +129,9 @@ plan("mex") = matlab.buildtool.tasks.MexTask.forEachFile(mexSourceFiles, mexOutp end ``` - - In the build file, we create a [`plan`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the it. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) API, introduced in R2025a, converts every C++ file within the specified folder into MEX functions. [`MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [`FileCollection`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input, which can be created using the [`files`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API. - + ## Expanding to create multiple MEX functions From fdae922ed00df392b0c788da3312cfe739daa3df Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 15 May 2025 16:55:25 +0530 Subject: [PATCH 058/105] Added a note for older release --- MEX.md | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/MEX.md b/MEX.md index 0ffba2a..fe9094f 100644 --- a/MEX.md +++ b/MEX.md @@ -129,14 +129,30 @@ plan("mex") = matlab.buildtool.tasks.MexTask.forEachFile(mexSourceFiles, mexOutp end ``` -In the build file, we create a [`plan`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the it. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) API, introduced in R2025a, converts every C++ file within the specified folder into MEX functions. [`MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [`FileCollection`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input, which can be created using the [`files`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API. +In the build file, we create a [`plan`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the it. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) API, introduced in R2025a, converts every C++ file within the specified folder into MEX functions. [`MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [`FileCollection`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input. The [`files`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API can be used to create a `FileCollection`. - + +The previously mentioned build file works only in MATLAB R2025a and later, for older releases of MATLAB you can use `mextask` given in the arithmetic toolbox repository. ## Expanding to create multiple MEX functions _Insert the add and subtract example here_ - +``` text +arithmetic/ +├───cpp/ +│ └───mexfunctions/ +| ├───addMex.cpp +│ └───invertMex.cpp +├───toolbox/ +| ├───private +| | ├───addMex.mexw64 +| | └───invertMex.mexw64 +| ├───add.m +| ├───subtract.m +| └───invertNumber.m +├───arithmetic.prj +└───buildfile.m +``` ### Multi platform MEX functions build using CI systems From a84f2ec78464b6f1c12ee16ce0bf98fb9f80c616 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 15 May 2025 17:35:06 +0530 Subject: [PATCH 059/105] Updated based on Rob's feedback --- MEX.md | 55 ++++++++++++++++++++++++------------------------------- 1 file changed, 24 insertions(+), 31 deletions(-) diff --git a/MEX.md b/MEX.md index fe9094f..414797b 100644 --- a/MEX.md +++ b/MEX.md @@ -5,28 +5,15 @@ ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) - - - -Welcome to the MATLAB® MEX Best Practice guide, this document extends [MATLAB Toolbox Best Practices](./README.md) by offering advise on integrating [MEX files](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) into your MATLAB toolbox projects. MEX function enable you to harness the power of _C/C++/Fortran_ functions within MATLAB, and this guide will help you with organizing the MEX source code files and MEX functions. - - +Welcome to the MATLAB® MEX Best Practice guide, this document extends [MATLAB Toolbox Best Practices](./README.md) by offering advise on integrating [MEX files](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) into your MATLAB toolbox projects. MEX functions enable you to harness the power of _C/C++/Fortran_ functions within MATLAB.In this document when we say "C++", we mean "C, C++, and Fortran." ## Overview -[MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) are compiler code that bridges the gap between MATLAB and C, C++, or Fortran. They behave like a MATLAB function and are platform dependent. You can determine the MEX extension for your platform using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This makes it easier for others to understand and contribute to your project. +[MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) are compiled code that bridges the gap between MATLAB and C++. They behave like a MATLAB function and you must build them for each operating system you want to run on. You can determine the MEX extension for your operating system using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This makes it easier for others to understand and contribute to your project. -To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. To make things clearer, when we say "C++", we mean "C, C++, and Fortran." For key concepts please refer to the [Toolbox Best Practices](./README.md). +To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. For key concepts please refer to the [Toolbox Best Practices](./README.md). - - - - ## Creating a MEX function from a single C++ source file - -Suppose you have a C++ MEX source file. This file has got its own MEX gateway and doesn't depend on any external library or other source files. You want to organize this file in a way that makes building, integrating and sharing the MEX function easy. According to the [Toolbox best practices](./README.md), MEX (C++) source files need not be distributed to the toolbox users, since they are not required to run the toolbox. So, here’s what we suggest: keep your MEX source file outside of the `toolbox` folder. You can do this by creating a folder called `cpp` at the `root` folder. Inside the `cpp` folder, make another folder named `mexfunctions`, and that’s where you put your MEX function’s source code. +Suppose you have a C++ MEX source file that has every thing you need in it. You want to organize this file in a way that makes building, integrating and sharing the MEX function easy. +MEX source files need not be distributed to the toolbox users, since they are not required to run the toolbox. +Our recommendation is that you keep the MEX source file outside of the `toolbox` folder. Create a folder called `cpp` at the root folder. Inside the `cpp` folder, make a folder with the same name as your MEX function, which contains the source code of your MEX function. + + +Let’s walk through how this organization works with the arithmetic toolbox example. We implement a MATLAB function `invertNumber()`, under the hood, this function uses the MEX functions `invertMex()`, which we compiled from `invertMex.cpp`. You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, with `invertMex` being the name of the MEX function. + +``` text +arithmetic/ +├───cpp/ +│ └───invertMex/ +│ └───invertMex.cpp +├───toolbox/ +| ├───add.m +| ├───subtract.m +| └───invertNumber.m +└───arithmetic.prj +``` + + -Let’s walk through how this organization works with the arithmetic toolbox example. We implement a MATLAB function `invertNumber()`, under the hood, this function uses the MEX functions `invertMex()`, which we compiled from `invertMex.cpp`. You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, with `invertMex` being the name of the MEX function. This naming convention is helpful for a couple of reasons: @@ -62,18 +66,7 @@ This naming convention is helpful for a couple of reasons: The organization of the MEX source files for the arithmetic toolbox relative to the toolbox folder is shown below: --> -``` text -arithmetic/ -├───cpp/ -│ └───mexfunctions/ -│ └───invertMex.cpp -├───toolbox/ -| ├───add.m -| ├───subtract.m -| └───invertNumber.m -├───arithmetic.prj -└───buildfile.m -``` + ### Building MEX functions -Suppose you have a C++ MEX source file that has every thing you need in it. You want to organize this file in a way that makes building, integrating and sharing the MEX function easy. +Suppose you have a C++ MEX source file that is self contained (has a MEX gateway and does not depend on other files). You want to organize this file in a way that makes building, integrating and sharing the MEX function easy. MEX source files need not be distributed to the toolbox users, since they are not required to run the toolbox. -Our recommendation is that you keep the MEX source file outside of the `toolbox` folder. Create a folder called `cpp` at the root folder. Inside the `cpp` folder, make a folder with the same name as your MEX function, which contains the source code of your MEX function. +We recommend keeping the MEX source file outside of the `toolbox` folder. - -Let’s walk through how this organization works with the arithmetic toolbox example. We implement a MATLAB function `invertNumber()`, under the hood, this function uses the MEX functions `invertMex()`, which we compiled from `invertMex.cpp`. You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, with `invertMex` being the name of the MEX function. +For the Arithmetic toolbox: +1. Create a folder called `cpp` at the root folder +2. Create a subfolder called `mexfunctions` within the `cpp` folder and save `invertMex.cpp` within this folder +3. Create a private folder within the toolbox folder for storing the compiled MEX functions ``` text arithmetic/ ├───cpp/ -│ └───invertMex/ +│ └───mexfunctions/ │ └───invertMex.cpp ├───toolbox/ +| ├───private/ | ├───add.m | ├───subtract.m | └───invertNumber.m └───arithmetic.prj ``` +You can use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, to compile `invertMex.cpp` into MEX functions. You need to provide the path of `invertMex.cpp` and path of the `private` folder as inputs to the `mex` command. + +```matlab +>>source = fullfile("cpp", "mexfunctions", "invertMex.cpp"); +>>destination = fullfile("toolbox", "private"); +>>mex(source, destination) +``` +After running the `mex` command, the folder structure of the repository would look: +``` text +arithmetic/ +├───cpp/ +│ └───mexfunctions/ +│ └───invertMex.cpp +├───toolbox/ +| ├───private/ +| | └───invertMex.mexw64 +| ├───add.m +| ├───subtract.m +| └───invertNumber.m +└───arithmetic.prj +``` + +The above command compiles `invertMex.cpp` and places the MEX functions in a `private` folder, we have made the MEX function [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). Our motivation for doing this is to restrict the toolbox user from calling these functions directly. Instead, we suggest always accessing MEX functions through a MATLAB script. This way, you have control over what gets passed as input to the MEX function, this helps avoid errors from unexpected or unhandled inputs. + + +Let’s walk through how this organization works with the arithmetic toolbox example. We implement a MATLAB function `invertNumber()`, under the hood, this function uses the MEX functions `invertMex()`, which we compiled from `invertMex.cpp`. You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, with `invertMex` being the name of the MEX function. + + @@ -71,14 +102,7 @@ BP: Removing the line above, TBP does not tell anything --> ### Building MEX functions -You can use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, to compile your MEX source code into MEX files. You need to provide the path to your MEX source file and where you want the compiled MEX file to go as inputs to the `mex` command. So, if you run the `mex` command from the `root` folder, you’d point to `cpp/mexfunctions/invertMex.cpp` as your source, and set `toolbox/private/` as the destination. -```matlab ->>source = fullfile("cpp", "mexfunctions", "invertMex.cpp"); ->>destination = fullfile("toolbox", "private"); ->>mex(source, destination) -``` -By placing the MEX file in a `private` folder, we have made the MEX function [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). Our motivation for doing this is to restrict the toolbox user from calling these functions directly. Instead, we suggest always accessing MEX functions through a MATLAB script. This way, you have control over what gets passed as input to the MEX function, this helps avoid errors from unexpected or unhandled inputs. Also, since MEX files are just build artifacts, it’s a good idea to leave them out of version control. You can do this by adding the `private` folder to your `.gitignore` file—there’s no need to track those files in your repo. ``` text From 58346ccafb97e8f42e394f41763535097bb0e892 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 20 May 2025 19:14:35 +0530 Subject: [PATCH 061/105] Included multiple source mex files --- MEX.md | 87 +++++++++++++++++++++++++++++----------------------------- 1 file changed, 44 insertions(+), 43 deletions(-) diff --git a/MEX.md b/MEX.md index 014d869..0c53e1b 100644 --- a/MEX.md +++ b/MEX.md @@ -28,15 +28,12 @@ BP: Introduced the buildtool later as suggested--> -Suppose you have a C++ MEX source file that is self contained (has a MEX gateway and does not depend on other files). You want to organize this file in a way that makes building, integrating and sharing the MEX function easy. -MEX source files need not be distributed to the toolbox users, since they are not required to run the toolbox. -We recommend keeping the MEX source file outside of the `toolbox` folder. +Suppose you have a C++ MEX source file that is self contained (has a MEX gateway and does not depend on other files). You want to organize this file in a way that makes building, integrating and sharing the MEX function easy. MEX source files need not be distributed to the toolbox users, since they are not required to run the toolbox. We recommend keeping the MEX source file outside of the `toolbox` folder. - -For the Arithmetic toolbox: -1. Create a folder called `cpp` at the root folder -2. Create a subfolder called `mexfunctions` within the `cpp` folder and save `invertMex.cpp` within this folder -3. Create a private folder within the toolbox folder for storing the compiled MEX functions +For the Arithmetic toolbox, create: +1. `cpp` folder at the root folder +2. `mexfunctions` subfolder within the `cpp` folder and save `invertMex.cpp` within this folder +3. `private` folder within the toolbox folder for storing the compiled MEX functions ``` text arithmetic/ ├───cpp/ @@ -56,7 +53,7 @@ You can use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) comm >>destination = fullfile("toolbox", "private"); >>mex(source, destination) ``` -After running the `mex` command, the folder structure of the repository would look: +The `mex` command would create the MEX function and place it within the `private` folder. ``` text arithmetic/ ├───cpp/ @@ -71,10 +68,16 @@ arithmetic/ └───arithmetic.prj ``` -The above command compiles `invertMex.cpp` and places the MEX functions in a `private` folder, we have made the MEX function [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). Our motivation for doing this is to restrict the toolbox user from calling these functions directly. Instead, we suggest always accessing MEX functions through a MATLAB script. This way, you have control over what gets passed as input to the MEX function, this helps avoid errors from unexpected or unhandled inputs. +By placing the MEX functions within the `private` folder, we have made it [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). Our motivation for doing this is to restrict the toolbox user from calling the MEX functions directly. We suggest accessing MEX functions through a MATLAB script. This way, you can control what gets passed as input to the MEX function, this prevents errors from unexpected or unhandled inputs. + +You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, with `invertMex` being the name of the MEX function. This naming convention is helpful for the following reasons: + +* It makes it easy to match each MEX function to its source code. +* By adding the "Mex" suffix, it’s immediately clear that this is a MEX function—not a regular MATLAB function. + +MEX files are just build artifacts, it’s a good idea to leave them out of version control. You can do this by adding the `private` folder to your `.gitignore` file. -Let’s walk through how this organization works with the arithmetic toolbox example. We implement a MATLAB function `invertNumber()`, under the hood, this function uses the MEX functions `invertMex()`, which we compiled from `invertMex.cpp`. You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, with `invertMex` being the name of the MEX function. @@ -89,10 +92,7 @@ RP: Be able to say "We are going to create the invertMex function, which on Wind BP: Added a statement similar to the one suggested--> -This naming convention is helpful for a couple of reasons: -* First, it makes it easy to match each MEX function to its source code. -* Second, by adding the "Mex" suffix, it’s immediately clear that this is a MEX function—not a regular MATLAB function. BP: Added the mex command based build and introduced the buildtool lated --> -Also, since MEX files are just build artifacts, it’s a good idea to leave them out of version control. You can do this by adding the `private` folder to your `.gitignore` file—there’s no need to track those files in your repo. ``` text arithmetic/ ├───cpp/ @@ -121,7 +120,7 @@ arithmetic/ ``` -### Automating using `buildtool` +### Automation using `buildtool` If you’ve got several MEX functions, running the `mex` command for each one can be tedious. In addition, you might also want to document the build recipe, so that you can refer to it in the future or shared with others. That’s where MATLAB’s [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b can be useful. Buildtool uses a build file to document the build recipe, which has several advantages: @@ -152,28 +151,6 @@ In the build file, we create a [`plan`](https://www.mathworks.com/help/matlab/re The previously mentioned build file works only in MATLAB R2025a and later, for older releases of MATLAB you can use `mextask` given in the arithmetic toolbox repository. -## Expanding to create multiple MEX functions -_Insert the add and subtract example here_ -``` text -arithmetic/ -├───cpp/ -│ └───mexfunctions/ -| ├───addMex.cpp -│ └───invertMex.cpp -├───toolbox/ -| ├───private -| | ├───addMex.mexw64 -| | └───invertMex.mexw64 -| ├───add.m -| ├───subtract.m -| └───invertNumber.m -├───arithmetic.prj -└───buildfile.m -``` - - -### Multi platform MEX functions build using CI systems - ## Creating a MEX function from multiple C++ source files @@ -197,6 +174,24 @@ arithmetic/ └───buildfile.m ``` + -The layout of the files with the `private` folder is shown below, for the sake of clarity we have shown the MEX functions for all the major platforms. Not that the GitHub repository for the Arithmetic toolbox will not contain the MEX functions since they are ignored by the version control system. You can create the MEX functions by running the following command from MATLAB terminal: + -``` matlab + + ### Out of process MEX host -If you are using MEX functions written using the C++, we recommend calling the MEX function from an out of process MEX host. You can create an out of process MEX host using the [mexhost](https://www.mathworks.com/help/matlab/ref/mexhost.html) command. It protects MATLAB from crashing due to unexpected errors originating from the MEX function execution. +For MEX functions written using the C++, we recommend calling the MEX function from an out of process MEX host. You can create an out of process MEX host using the [mexhost](https://www.mathworks.com/help/matlab/ref/mexhost.html) command. It protects MATLAB from crashing due to unexpected errors originating from the MEX function execution. + +**Caution**: mexhost does not support MEX functions written in C and Fortran. ## Incorporating External Libraries @@ -418,6 +416,9 @@ arithmetic/ └───buildfile.m ``` --> + +## Multi platform MEX functions build using CI systems + ## Testing Develop tests for the MATLAB layer in the `tests` folder. While MEX functionality is indirectly tested through MATLAB scripts, this approach ensures comprehensive validation of your toolbox's capabilities. From 1879dba1bf478ad34496e4cfcb823a6adefddc1e Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Tue, 20 May 2025 13:31:33 -0400 Subject: [PATCH 062/105] Test that rob can push changes. --- MEX.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/MEX.md b/MEX.md index 0c53e1b..945f996 100644 --- a/MEX.md +++ b/MEX.md @@ -5,6 +5,8 @@ ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) +Test that Rob can push changes + Welcome to the MATLAB® MEX Best Practice guide, this document extends [MATLAB Toolbox Best Practices](./README.md) by offering advise on integrating [MEX files](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) into your MATLAB toolbox projects. MEX functions enable you to harness the power of _C/C++/Fortran_ functions within MATLAB.In this document when we say "C++", we mean "C, C++, and Fortran." From 6157996c5450dade6088f5152f88a2a80c0d02ec Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Tue, 20 May 2025 13:32:12 -0400 Subject: [PATCH 063/105] Yes, Rob can push changes --- MEX.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/MEX.md b/MEX.md index 945f996..c48e19f 100644 --- a/MEX.md +++ b/MEX.md @@ -5,9 +5,6 @@ ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) -Test that Rob can push changes - - Welcome to the MATLAB® MEX Best Practice guide, this document extends [MATLAB Toolbox Best Practices](./README.md) by offering advise on integrating [MEX files](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) into your MATLAB toolbox projects. MEX functions enable you to harness the power of _C/C++/Fortran_ functions within MATLAB.In this document when we say "C++", we mean "C, C++, and Fortran." ## Overview From 1e840461ec76b99cafc1f36a2da9b794f60ba246 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Wed, 21 May 2025 14:22:04 +0530 Subject: [PATCH 064/105] minor updates --- MEX.md | 1 - 1 file changed, 1 deletion(-) diff --git a/MEX.md b/MEX.md index 0c53e1b..c48e19f 100644 --- a/MEX.md +++ b/MEX.md @@ -5,7 +5,6 @@ ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) - Welcome to the MATLAB® MEX Best Practice guide, this document extends [MATLAB Toolbox Best Practices](./README.md) by offering advise on integrating [MEX files](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) into your MATLAB toolbox projects. MEX functions enable you to harness the power of _C/C++/Fortran_ functions within MATLAB.In this document when we say "C++", we mean "C, C++, and Fortran." ## Overview From 688e5b76b76da3553c3766d991c300c052a51e0a Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Wed, 21 May 2025 17:12:01 +0530 Subject: [PATCH 065/105] Added the build file for multiple source MEX functions --- MEX.md | 62 +++++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 46 insertions(+), 16 deletions(-) diff --git a/MEX.md b/MEX.md index c48e19f..2b503fa 100644 --- a/MEX.md +++ b/MEX.md @@ -30,9 +30,9 @@ BP: Added the single source scenario, first. Suppose you have a C++ MEX source file that is self contained (has a MEX gateway and does not depend on other files). You want to organize this file in a way that makes building, integrating and sharing the MEX function easy. MEX source files need not be distributed to the toolbox users, since they are not required to run the toolbox. We recommend keeping the MEX source file outside of the `toolbox` folder. For the Arithmetic toolbox, create: -1. `cpp` folder at the root folder +1. `cpp` folder under the root folder 2. `mexfunctions` subfolder within the `cpp` folder and save `invertMex.cpp` within this folder -3. `private` folder within the toolbox folder for storing the compiled MEX functions +3. `private` folder within the `toolbox` folder for storing the compiled MEX functions ``` text arithmetic/ ├───cpp/ @@ -45,6 +45,9 @@ arithmetic/ | └───invertNumber.m └───arithmetic.prj ``` +### Building MEX functions + You can use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, to compile `invertMex.cpp` into MEX functions. You need to provide the path of `invertMex.cpp` and path of the `private` folder as inputs to the `mex` command. ```matlab @@ -67,14 +70,14 @@ arithmetic/ └───arithmetic.prj ``` -By placing the MEX functions within the `private` folder, we have made it [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). Our motivation for doing this is to restrict the toolbox user from calling the MEX functions directly. We suggest accessing MEX functions through a MATLAB script. This way, you can control what gets passed as input to the MEX function, this prevents errors from unexpected or unhandled inputs. +By placing the MEX functions within the `private` folder, we have made it [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). By doing this, you restrict your users from calling the MEX functions directly. We suggest accessing MEX functions through a MATLAB script. This way, you can control what gets passed as input to the MEX function, there by preventing errors from unexpected or unhandled inputs. You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, with `invertMex` being the name of the MEX function. This naming convention is helpful for the following reasons: * It makes it easy to match each MEX function to its source code. * By adding the "Mex" suffix, it’s immediately clear that this is a MEX function—not a regular MATLAB function. -MEX files are just build artifacts, it’s a good idea to leave them out of version control. You can do this by adding the `private` folder to your `.gitignore` file. +We recommend leaving the MEX files out of version control since they are just build artifacts. You can do this by adding the `private` folder to your `.gitignore` file. @@ -98,12 +101,10 @@ The organization of the MEX source files for the arithmetic toolbox relative to BP: Removing the line above, TBP does not tell anything --> -### Building MEX functions - -``` text + + ### Automation using `buildtool` @@ -127,7 +128,7 @@ If you’ve got several MEX functions, running the `mex` command for each one ca BP: Moved the motivation for buildfile --> 1. *Incremental build:* By using `MexTask` instead of the `mex` command, you automatically get support for incremental build, i.e) buildtool only re-compiles the files that have changed. -2. *No hardcoding of source files:* Since the build file references to the folder containing the MEX source files and not the source files directly, the build file can scale to handle any number of single-source MEX functions, as long as the source files are placed within the cpp/mexfunctions folder. +2. *No hardcoding of source files:* Since the build file refers to the folder containing the MEX source files and not the source files directly, the build file can scale to handle any number of single-source MEX functions, as long as the source files are placed within the cpp/mexfunctions folder. 3. *Platform independence:* Although MEX functions themselves are platform dependent, the build file does not rely on any platform-specific information to compile them, making it possible to use the same build file across different platforms. @@ -148,15 +149,16 @@ In the build file, we create a [`plan`](https://www.mathworks.com/help/matlab/re -The previously mentioned build file works only in MATLAB R2025a and later, for older releases of MATLAB you can use `mextask` given in the arithmetic toolbox repository. +The previously mentioned build file works only in MATLAB R2025a and later, for older releases of MATLAB you can use `mextask` as shown in the arithmetic toolbox repository. ## Creating a MEX function from multiple C++ source files -It is common to have MEX functions generated from multiple C++ source files, we call these MEX functions multiple source MEX functions. For such MEX functions one of the source files implement the gateway function and other source files implement methods that are called within the gateway function. For multiple source MEX functions, create a folder for each MEX function within the `cpp` folder. The name of the folder should be same as that of the MEX function that will be created out for source code with the folder. The folder name can be suffixed with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. - +You can create MEX functions from multiple C++ source files, we call them multiple source MEX functions. For such MEX functions one of the source files implement the gateway function and other files implement methods that are called within the gateway function. You can organize the source code for all these MEX functions under a single folder -For the sake of discussion let us implement substractMex MEXfunction as a Multiple source MEX function. The organization of the source files is shown below. +- Create a folder for each MEX function within the `cpp` folder. +- Name of the folder should be same as that of the MEX function +- Folder name is suffixed with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. ``` text arithmetic/ @@ -165,14 +167,42 @@ arithmetic/ │ │ ├───substract.cpp % Implements gateway function │ │ └───substractImp.cpp % other features │ └───mexfunctions/ -│ └───addMex.cpp +│ └───invertMex.cpp ├───toolbox/ | ├───add.m -| └───subtract.m +| ├───subtract.m +| └───invertNumber.m ├───arithmetic.prj └───buildfile.m ``` +```matlab +function plan = buildfile + +plan = buildplan(); + +mexOutputFolder = fullfile("toolbox","private"); + +% Compile all the .cpp files inside cpp/mexfunctions into MEX functions +mexSourceFiles = files(plan, fullfile("cpp", "mexfunctions", "*.cpp")); +plan("mex") = matlab.buildtool.tasks.MexTask.forEachFile(mexSourceFiles, mexOutputFolder); + +% Compile all the folders inside cpp/*Mex into MEX functions +foldersToMex = plan.files("cpp/*Mex").select(@isfolder); +for f = foldersToMex.paths + [~, folderName] = fileparts(f); + plan("mex:"+folderName) = matlab.buildtool.tasks.MexTask(fullfile(f, "**/*.cpp"), ... + mexOutputFolder, ... + Options="-I""cpp/includes""", ... + Filename=folderName); +end +plan("mex").Description = "Build MEX functions"; +end +``` + + + + @@ -154,7 +157,7 @@ The previously mentioned build file works only in MATLAB R2025a and later, for o ## Creating a MEX function from multiple C++ source files -You can create MEX functions from multiple C++ source files, we call them multiple source MEX functions. For such MEX functions one of the source files implement the gateway function and other files implement methods that are called within the gateway function. You can organize the source code for all these MEX functions under a single folder +You can create MEX functions from multiple C++ source files, we call them multiple source MEX functions. For such MEX functions one of the source files implement the gateway function and other files implement methods that are called within the gateway function. You can organize the source code for all these MEX functions under a single folder. - Create a folder for each MEX function within the `cpp` folder. - Name of the folder should be same as that of the MEX function @@ -201,8 +204,6 @@ end ``` - - You can use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, to compile `invertMex.cpp` into MEX functions. You need to provide the path of `invertMex.cpp` and path of the `private` folder as inputs to the `mex` command. ```matlab ->>source = fullfile("cpp", "mexfunctions", "invertMex.cpp"); +>>source = fullfile("cpp", "invertMex", "*.cpp"); >>destination = fullfile("toolbox", "private"); ->>mex(source, destination) +>>mex(source, destination) % Add output file name ``` The `mex` command would create the MEX function and place it within the `private` folder. ``` text arithmetic/ ├───cpp/ -│ └───mexfunctions/ +│ └───invertMex/ │ └───invertMex.cpp ├───toolbox/ | ├───private/ | | └───invertMex.mexw64 -| ├───add.m -| ├───subtract.m +| ├... | └───invertNumber.m └───arithmetic.prj ``` + ## Why put the MEX functions within a private folder? + By putting it in a [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html) folder, you restrict your users from calling the MEX functions directly. It is very common for MEX files to crash MATLAB if they get unexpected inputs. By limiting access to MEX functions from a MATLAB function, you can control what gets passed as input to the MEX function, there by preventing errors from unexpected or unhandled inputs. -By placing the MEX functions within the `private` folder, we have made it [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html). By doing this, you restrict your users from calling the MEX functions directly. We suggest accessing MEX functions through a MATLAB script. This way, you can control what gets passed as input to the MEX function, there by preventing errors from unexpected or unhandled inputs. +The MATLAB function `invertNumber` within the `toolbox` folder makes a call to the `invertMex` MEX function. We suggest adding an [arguments block](https://www.mathworks.com/help/matlab/ref/arguments.html) to `invertNumber` to validate the inputs before passing it to `invertMex`. + You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, with `invertMex` being the name of the MEX function. This naming convention is helpful for the following reasons: * It makes it easy to match each MEX function to its source code. * By adding the "Mex" suffix, it’s immediately clear that this is a MEX function—not a regular MATLAB function. -The m-script `invertNumber.m` within the `toolbox` folder makes a call to the `invertMex` MEX function. We suggest adding an [arguments block](https://www.mathworks.com/help/matlab/ref/arguments.html) to `invertNumber.m` for validating the inputs before passing it to `invertMex`. - - -We recommend leaving the MEX files out of version control since they are just build artifacts. You can do this by adding the `private` folder to your `.gitignore` file. - - - - - - - - - - - - - - - - - - - - ### Automation using `buildtool` - -If you’ve got several MEX functions, running the `mex` command for each one can be tedious. In addition, you might also want to document the build recipe, so that you can refer to it in the future or shared with others. That’s where MATLAB’s [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b can be useful. Buildtool uses a build file to document the build recipe, which has several advantages: - - -1. *Incremental build:* By using `MexTask` instead of the `mex` command, you automatically get support for incremental build, i.e) buildtool only re-compiles the files that have changed. -2. *No hardcoding of source files:* Since the build file refers to the folder containing the MEX source files and not the source files directly, the build file can scale to handle any number of single-source MEX functions, as long as the source files are placed within the cpp/mexfunctions folder. -3. *Platform independence:* Although MEX functions themselves are platform dependent, the build file does not rely on any platform-specific information to compile them, making it possible to use the same build file across different platforms. - +Running the `mex` command for each MEX file can be tedious and error prone. That’s where MATLAB’s [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b can be useful. The following `buildfile.m` creates a `mex` task, that builds your MEX files. ``` matlab function plan = buildfile @@ -144,24 +91,21 @@ plan = buildplan(); % Compile all the .cpp files inside cpp/mexfunctions into MEX functions mexSourceFiles = files(plan, fullfile("cpp", "mexfunctions", "*.cpp")); mexOutputFolder = fullfile("toolbox","private"); -plan("mex") = matlab.buildtool.tasks.MexTask.forEachFile(mexSourceFiles, mexOutputFolder); - +plan("mex") = matlab.buildtool.tasks.MexTask(mexSourceFiles, mexOutputFolder); +% Make this work in 24a end ``` -In the build file, we create a [`plan`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan-class.html) and add a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) to the it. The [`matlab.buildtool.tasks.MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) API, introduced in R2025a, converts every C++ file within the specified folder into MEX functions. [`MexTask.forEachFile`](https://www-jobarchive.mathworks.com/Bdoc/latest_pass/matlab/help/matlab/ref/matlab.buildtool.tasks.mextask.foreachfile.html) takes a [`FileCollection`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.io.filecollection-class.html) as input. The [`files`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.plan.files.html) API can be used to create a `FileCollection`. - + - -The previously mentioned build file works only in MATLAB R2025a and later, for older releases of MATLAB you can use `mextask` as shown in the arithmetic toolbox repository. ## Creating a MEX function from multiple C++ source files -You can create MEX functions from multiple C++ source files, we call them multiple source MEX functions. For such MEX functions one of the source files implement the gateway function and other files implement methods that are called within the gateway function. You can organize the source code for all these MEX functions under a single folder. +Tha above pattern naturally extends to MEX functions that has multiple C++ source files. One of the source files must contain the [gateway](https://www.mathworks.com/help/matlab/matlab_external/gateway-routine.html) function. Place a the source files under a single folder. -- Create a folder for each MEX function within the `cpp` folder. + ``` text arithmetic/ @@ -169,39 +113,17 @@ arithmetic/ │ ├───substractMex/ │ │ ├───substract.cpp % Implements gateway function │ │ └───substractImp.cpp % other features -│ └───mexfunctions/ +│ └───invertMex/ │ └───invertMex.cpp ├───toolbox/ -| ├───add.m +| ├─... | ├───subtract.m | └───invertNumber.m ├───arithmetic.prj └───buildfile.m ``` -```matlab -function plan = buildfile - -plan = buildplan(); - -mexOutputFolder = fullfile("toolbox","private"); - -% Compile all the .cpp files inside cpp/mexfunctions into MEX functions -mexSourceFiles = files(plan, fullfile("cpp", "mexfunctions", "*.cpp")); -plan("mex") = matlab.buildtool.tasks.MexTask.forEachFile(mexSourceFiles, mexOutputFolder); - -% Compile all the folders inside cpp/*Mex into MEX functions -foldersToMex = plan.files("cpp/*Mex").select(@isfolder); -for f = foldersToMex.paths - [~, folderName] = fileparts(f); - plan("mex:"+folderName) = matlab.buildtool.tasks.MexTask(fullfile(f, "**/*.cpp"), ... - mexOutputFolder, ... - Options="-I""cpp/includes""", ... - Filename=folderName); -end -plan("mex").Description = "Build MEX functions"; -end -``` +The `buildfile.m` is the same as before. + ### Automation using `buildtool` Running the `mex` command for each MEX file can be tedious and error prone. That’s where MATLAB’s [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b can be useful. The following `buildfile.m` creates a `mex` task, that builds your MEX files. @@ -125,124 +132,7 @@ arithmetic/ The `buildfile.m` is the same as before. - - - - - - - - - - -### Out of process MEX host -For MEX functions written using the C++, we recommend calling the MEX function from an out of process MEX host. You can create an out of process MEX host using the [mexhost](https://www.mathworks.com/help/matlab/ref/mexhost.html) command. It protects MATLAB from crashing due to unexpected errors originating from the MEX function execution. - -**Caution**: mexhost does not support MEX functions written in C and Fortran. +## Special case: Many MEX functions with a single C++ source file ## Incorporating External Libraries From 4896681cc0476b961a48c846c708f0032667bbbe Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 22 May 2025 13:01:38 +0530 Subject: [PATCH 069/105] updated the build script --- MEX.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/MEX.md b/MEX.md index f53df23..a4d9d66 100644 --- a/MEX.md +++ b/MEX.md @@ -89,17 +89,23 @@ For MEX functions written using the C++, we recommend calling the MEX function f ### Automation using `buildtool` Running the `mex` command for each MEX file can be tedious and error prone. That’s where MATLAB’s [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b can be useful. The following `buildfile.m` creates a `mex` task, that builds your MEX files. -``` matlab +```matlab function plan = buildfile - -% Create a plan +% Should work in 24a plan = buildplan(); -% Compile all the .cpp files inside cpp/mexfunctions into MEX functions -mexSourceFiles = files(plan, fullfile("cpp", "mexfunctions", "*.cpp")); mexOutputFolder = fullfile("toolbox","private"); -plan("mex") = matlab.buildtool.tasks.MexTask(mexSourceFiles, mexOutputFolder); -% Make this work in 24a + +% Compile all the folders inside cpp/*Mex into MEX functions +foldersToMex = plan.files("cpp/*Mex").select(@isfolder); +for f = foldersToMex.paths + [~, folderName] = fileparts(f); + plan("mex:"+folderName) = matlab.buildtool.tasks.MexTask(fullfile(f, "**/*.cpp"), ... + mexOutputFolder, ... + Filename=folderName); +end +plan("mex").Description = "Build MEX functions"; + end ``` From d9d31314d2908feed52c1b67555078ee0c8c7ac3 Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Thu, 22 May 2025 10:22:42 -0400 Subject: [PATCH 070/105] Reviewed and revised up to and including buildtool. Added todo list --- MEX.md | 130 ++++++++++++++++++++++++++++++--------------------------- 1 file changed, 68 insertions(+), 62 deletions(-) diff --git a/MEX.md b/MEX.md index a4d9d66..e9b5872 100644 --- a/MEX.md +++ b/MEX.md @@ -3,57 +3,57 @@ > [!IMPORTANT] > This page is under construction and review +**TODO:** + +- [X] Review and revise up to and including buildtool. +- [ ] Update `buildfile.m` in buildtool section +- [ ] Review and update multiple source file section +- [ ] Review and update `mexfunction` folder scenario section +- [ ] Review and update external libraries section +- [ ] Review and update CI / GitHub Actions section + ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) -Welcome to the MATLAB® MEX Best Practice guide, this document extends [MATLAB Toolbox Best Practices](./README.md) by offering advise on integrating [MEX files](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) into your MATLAB toolbox projects. MEX functions enable you to harness the power of _C/C++/Fortran_ functions within MATLAB.In this document when we say "C++", we mean "C, C++, and Fortran." +Welcome to the MATLAB® MEX Best Practice guide, which extends [MATLAB Toolbox Best Practices](./README.md) focusing on integrating [MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) into your MATLAB toolbox. MEX functions enable you to harness the power of C, C++, and Fortran code within MATLAB. In this document when we say "C++", we mean "C, C++, and Fortran." ## Overview -[MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) are compiled code that bridges the gap between MATLAB and C++. They behave like a MATLAB function and you must build them for each operating system you want to run on. You can determine the MEX extension for your operating system using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). While the integration of MEX files can be intricate, this guide will navigate you through the process, ensuring smooth implementation in both development and production environments. This makes it easier for others to understand and contribute to your project. +[MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) are compiled functions that bridge the gap between MATLAB and C++. They behave like a MATLAB function and you must build them for each operating system you want to run on. You can determine the MEX file extension (for example, `.mexw64` in Microsoft Windows) for your operating system using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). This guide will navigate you through the process of integrating of MEX functions, ensuring smooth implementation in both development and production environments. This makes it easier for others to understand and contribute to your toolbox. -To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this project throughout the guide to demonstrate practical applications of these principles. For key concepts please refer to the [Toolbox Best Practices](./README.md). +To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this toolbox throughout the guide to demonstrate practical applications of these principles. For key concepts please refer to the [Toolbox Best Practices](./README.md). - - - - - - - - +## MEX function from a single C++ source file +Suppose you have a C++ MEX source file that is a single C++ source file. MEX source files need not be distributed to the toolbox users, since they are not required to run the toolbox. Only the compiled function needs to be distributed. We recommend keeping the C++ MEX source files outside of the `toolbox` folder in a folder focused on C++ code, `cpp`. -## Creating a MEX function from a single C++ source file - -Suppose you have a C++ MEX source file that is self contained (has a MEX gateway and does not depend on other files). You want to organize this file in a way that makes building, integrating and sharing the MEX function easy. MEX source files need not be distributed to the toolbox users, since they are not required to run the toolbox. We recommend keeping the MEX source file outside of the `toolbox` folder. +For each MEX function in the `cpp` folder, create a folder with the name that matches the name of your MEX function. This folder should end with `Mex` to indicate that all the files within the folder are associated with a single MEX function. + +Our example toolbox has a `cpp` folder in the root folder, with an `invertMex` subfolder inside. The C++ code `invertMex.cpp` is within this folder. While the `.cpp` file does not have to match the folder name, it can be convenient to do this in simple cases. -For the Arithmetic toolbox, create: -1. `cpp` folder under the root folder -2. `mexfunctions` subfolder within the `cpp` folder and save `invertMex.cpp` within this folder -3. `private` folder within the `toolbox` folder for storing the compiled MEX functions ``` text arithmetic/ ├───cpp/ │ └───invertMex/ │ └───invertMex.cpp ├───toolbox/ -| ├───private/ -| └───... +... └───arithmetic.prj ``` ### Building MEX functions - -You can use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, to compile `invertMex.cpp` into MEX functions. You need to provide the path of `invertMex.cpp` and path of the `private` folder as inputs to the `mex` command. + +You can use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, to compile `invertMex.cpp` into MEX functions. We suggest you place your compiled MEX functions in your `private` folder within the `toolbox` folder (see below). You need to provide the path of `invertMex.cpp` and path of the `private` folder as inputs to the `mex` command. ```matlab >>source = fullfile("cpp", "invertMex", "*.cpp"); >>destination = fullfile("toolbox", "private"); ->>mex(source, destination) % Add output file name +>>mex(source, "-outdir", destination, "-output", "invertMex") ``` The `mex` command would create the MEX function and place it within the `private` folder. + +Our example toolbox has: + +1. A `invertMex.mexw64` file in the `toolbox/private` folder +2. The MATLAB function `invertNumber.m` within the `toolbox` folder calls into the `invertMex` MEX function. We recommend using an [arguments block](https://www.mathworks.com/help/matlab/ref/arguments.html) in `invertNumber` to validate the inputs before passing it to `invertMex`. + ``` text arithmetic/ ├───cpp/ @@ -66,51 +66,51 @@ arithmetic/ | └───invertNumber.m └───arithmetic.prj ``` - ## Why put the MEX functions within a private folder? - By putting it in a [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html) folder, you restrict your users from calling the MEX functions directly. It is very common for MEX files to crash MATLAB if they get unexpected inputs. By limiting access to MEX functions from a MATLAB function, you can control what gets passed as input to the MEX function, there by preventing errors from unexpected or unhandled inputs. - -The MATLAB function `invertNumber` within the `toolbox` folder makes a call to the `invertMex` MEX function. We suggest adding an [arguments block](https://www.mathworks.com/help/matlab/ref/arguments.html) to `invertNumber` to validate the inputs before passing it to `invertMex`. - - -You’ll notice that we deliberately name the source file and the MEX function the same way. For example, on Windows, `invertMex.cpp` gets compiled into `invertMex.mexw64`, with `invertMex` being the name of the MEX function. This naming convention is helpful for the following reasons: - -* It makes it easy to match each MEX function to its source code. -* By adding the "Mex" suffix, it’s immediately clear that this is a MEX function—not a regular MATLAB function. - -If you are using git, we recommend leaving the compiled MEX files out of version control. You can add `*.mex*` to you `.gitignore` file. This is part of the [standard .gitignore](https://github.com/mathworks/gitignore/blob/main/Global/MATLAB.gitignore) for MATLAB. - -### Out of process MEX host -For MEX functions written using the C++, we recommend calling the MEX function from an out of process MEX host. You can create an out of process MEX host using the [mexhost](https://www.mathworks.com/help/matlab/matlab_external/out-of-process-execution-of-c-mex-functions.html) command. It protects MATLAB from crashing due to unexpected errors originating from the MEX function execution. - -**Note**: `mexhost` is only supported for C++ MEX functions. - - + ### Additional Notes + * **Why put the MEX functions within a private folder?** By putting it in a [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html) folder, you restrict your users from calling the MEX function directly. Even minor errors in a MEX function will crash MATLAB, especially if they receive unexpected inputs. By limiting access to MEX functions to a MATLAB function that you control, you ensure that only what you expect will be passed as input to the MEX function, preventing errors from unexpected or unhandled inputs. + * **Out of process MEX host** We recommend [Out-of-Process Execution of C++ MEX Functions](https://www.mathworks.com/help/matlab/matlab_external/out-of-process-execution-of-c-mex-functions.html). This isolates the MATLAB process from crashes in you C++ MEX function and allows you to use some third-party libraries that are not compatible with MATLAB. Use the [mexhost](https://www.mathworks.com/help/matlab/ref/mexhost.html) command to do this. Note that `mexhost` is only supported for C++ MEX functions. + * **Using git** In git source control systems, we recommend that you do not keep compiled MEX functions under version control, as they are derived files. Add `*.mex*` to your `.gitignore` file. This is part of the [standard .gitignore file](https://github.com/mathworks/gitignore/blob/main/Global/MATLAB.gitignore) for MATLAB. ### Automation using `buildtool` -Running the `mex` command for each MEX file can be tedious and error prone. That’s where MATLAB’s [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b can be useful. The following `buildfile.m` creates a `mex` task, that builds your MEX files. + +Running the `mex` command for each MEX function can be tedious and error prone. MATLAB’s [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b, can automate this and many other repetative processes for you. The following `buildfile.m` creates a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html), introduced in R2024a, that builds your MEX functions. ```matlab function plan = buildfile -% Should work in 24a -plan = buildplan(); - -mexOutputFolder = fullfile("toolbox","private"); - -% Compile all the folders inside cpp/*Mex into MEX functions -foldersToMex = plan.files("cpp/*Mex").select(@isfolder); -for f = foldersToMex.paths - [~, folderName] = fileparts(f); - plan("mex:"+folderName) = matlab.buildtool.tasks.MexTask(fullfile(f, "**/*.cpp"), ... - mexOutputFolder, ... - Filename=folderName); -end -plan("mex").Description = "Build MEX functions"; +% !!Revise to work in 24a!! + plan = buildplan(); + + mexOutputFolder = fullfile("toolbox","private"); + + % Compile all the folders inside cpp/*Mex into MEX functions + foldersToMex = plan.files("cpp/*Mex").select(@isfolder); + for f = foldersToMex.paths + [~, folderName] = fileparts(f); + plan("mex:"+folderName) = matlab.buildtool.tasks.MexTask(fullfile(f, "**/*.cpp"), ... + mexOutputFolder, ... + Filename=folderName); + end + plan("mex").Description = "Build MEX functions"; end ``` +Our example toolbox adds a `buildfile.m` to automate the building of the MEX functions: +``` text +arithmetic/ +├───cpp/ +│ └───invertMex/ +│ └───invertMex.cpp +├───toolbox/ +| ├───private/ +| | └───invertMex.mexw64 +| ├... +| ├───buildfile.m +| └───invertNumber.m +└───arithmetic.prj +``` ## Creating a MEX function from multiple C++ source files @@ -142,6 +142,12 @@ The `buildfile.m` is the same as before. ## Incorporating External Libraries + + + + + + You can call libraries implemented in C++ using MEX functions. Since MEX source files are just C++ source code, they use the syntax of C++ to access external libraries. You may be wondering where to store these external libraries? The answer to this question depends on the external library and the operating system with which you are working with. There are two broad categories of libraries: compile time and execution time libraries. @@ -306,7 +312,7 @@ arithmetic/ ## Conclusion -By following to these best practices, you'll create a robust, maintainable, and user-friendly MATLAB toolbox that harnesses the full potential of MEX files. Through effective project structure organization, build automation, and dependency management, you can focus on delivering great solutions to your users. +By following to these best practices, you'll create a robust, maintainable, and user-friendly MATLAB toolbox that harnesses the full potential of MEX functions. Through effective project structure organization, build automation, and dependency management, you can focus on delivering great solutions to your users. We welcome your input! For further details, suggestions, or to contribute, please [open an issue](https://github.com/mathworks/toolboxdesign/issues/new/choose). From 6f20d2d82bfb1f81687562754fca54a541a6c7b7 Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Thu, 22 May 2025 10:24:25 -0400 Subject: [PATCH 071/105] updated todo --- MEX.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index e9b5872..f027778 100644 --- a/MEX.md +++ b/MEX.md @@ -5,7 +5,9 @@ **TODO:** -- [X] Review and revise up to and including buildtool. +- [X] Review and revise overview. +- [X] Review and revise single source file section. +- [X] Review and revise buildtool section. - [ ] Update `buildfile.m` in buildtool section - [ ] Review and update multiple source file section - [ ] Review and update `mexfunction` folder scenario section From f3360396daa66c5220b8296f9028695a732bc1d0 Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Fri, 23 May 2025 11:38:51 -0400 Subject: [PATCH 072/105] Revised Creating a MEX function from multiple C++ source files --- MEX.md | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/MEX.md b/MEX.md index f027778..a0881ab 100644 --- a/MEX.md +++ b/MEX.md @@ -9,7 +9,7 @@ - [X] Review and revise single source file section. - [X] Review and revise buildtool section. - [ ] Update `buildfile.m` in buildtool section -- [ ] Review and update multiple source file section +- [X] Review and update multiple source file section - [ ] Review and update `mexfunction` folder scenario section - [ ] Review and update external libraries section - [ ] Review and update CI / GitHub Actions section @@ -79,7 +79,7 @@ Running the `mex` command for each MEX function can be tedious and error prone. ```matlab function plan = buildfile -% !!Revise to work in 24a!! +% !!Revise to work in 24a!! Make sure it supports multiple *Mex directoriesa and multiple .cpp files plan = buildplan(); mexOutputFolder = fullfile("toolbox","private"); @@ -122,17 +122,22 @@ Tha above pattern naturally extends to MEX functions that has multiple C++ sourc - Name of the folder should be same as that of the MEX function - Folder name is suffixed with 'Mex' to indicate that the contents of the folder should be compiled into a single MEX function. --> +Our example toolbox adds two .cpp files in the `subtractMex` folder, which builds to the `subtractMex.mexw64` binary in the `private` folder. `subtractNumber.m` provides a user accessible version that error checks: + ``` text arithmetic/ ├───cpp/ -│ ├───substractMex/ -│ │ ├───substract.cpp % Implements gateway function -│ │ └───substractImp.cpp % other features +│ ├───subtractMex/ +│ │ ├───subtract.cpp % Implements gateway function +│ │ └───subtractImpl.cpp % other features │ └───invertMex/ │ └───invertMex.cpp ├───toolbox/ +| ├───private/ +| | ├───invertMex.mexw64 +| | └───subtractMex.mexw64 | ├─... -| ├───subtract.m +| ├───subtractNumber.m | └───invertNumber.m ├───arithmetic.prj └───buildfile.m @@ -140,8 +145,7 @@ arithmetic/ The `buildfile.m` is the same as before. -## Special case: Many MEX functions with a single C++ source file - +## Many MEX functions with a single C++ source file ## Incorporating External Libraries From 8df53ddcfa321d35b8cf43a0d7773f9b1260d374 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 3 Jun 2025 14:38:06 +0530 Subject: [PATCH 073/105] minor updates --- MEX.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/MEX.md b/MEX.md index a0881ab..b0b438c 100644 --- a/MEX.md +++ b/MEX.md @@ -25,7 +25,7 @@ Welcome to the MATLAB® MEX Best Practice guide, which extends [MATLAB Toolbo To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this toolbox throughout the guide to demonstrate practical applications of these principles. For key concepts please refer to the [Toolbox Best Practices](./README.md). ## MEX function from a single C++ source file -Suppose you have a C++ MEX source file that is a single C++ source file. MEX source files need not be distributed to the toolbox users, since they are not required to run the toolbox. Only the compiled function needs to be distributed. We recommend keeping the C++ MEX source files outside of the `toolbox` folder in a folder focused on C++ code, `cpp`. +Suppose you have a single C++ MEX source file, this MEX source file need not be distributed to the toolbox users, since they are not required to run the toolbox. Only the compiled function needs to be distributed. We recommend keeping the C++ MEX source files outside of the `toolbox` folder in a folder focused on C++ code, `cpp`. For each MEX function in the `cpp` folder, create a folder with the name that matches the name of your MEX function. This folder should end with `Mex` to indicate that all the files within the folder are associated with a single MEX function. @@ -45,9 +45,9 @@ arithmetic/ You can use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, to compile `invertMex.cpp` into MEX functions. We suggest you place your compiled MEX functions in your `private` folder within the `toolbox` folder (see below). You need to provide the path of `invertMex.cpp` and path of the `private` folder as inputs to the `mex` command. ```matlab ->>source = fullfile("cpp", "invertMex", "*.cpp"); ->>destination = fullfile("toolbox", "private"); ->>mex(source, "-outdir", destination, "-output", "invertMex") +>> source = fullfile("cpp", "invertMex", "*.cpp"); +>> destination = fullfile("toolbox", "private"); +>> mex(source, "-outdir", destination, "-output", "invertMex") ``` The `mex` command would create the MEX function and place it within the `private` folder. @@ -79,7 +79,7 @@ Running the `mex` command for each MEX function can be tedious and error prone. ```matlab function plan = buildfile -% !!Revise to work in 24a!! Make sure it supports multiple *Mex directoriesa and multiple .cpp files +% !!Revise to work in 24a!! Make sure it supports multiple *Mex directories and multiple .cpp files plan = buildplan(); mexOutputFolder = fullfile("toolbox","private"); From 4f73ed0943e7794370eade1a0d92b1dc63206a4e Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 3 Jun 2025 15:12:53 +0530 Subject: [PATCH 074/105] Added the folder structure and buildfile for multiple single source MEX functions --- MEX.md | 39 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index b0b438c..df1d002 100644 --- a/MEX.md +++ b/MEX.md @@ -145,7 +145,44 @@ arithmetic/ The `buildfile.m` is the same as before. -## Many MEX functions with a single C++ source file +## Multiple single source MEX functions + +``` text +arithmetic/ +├───cpp/ +│ └───mexfunctions/ +| ├───powerMex.cpp +│ └───divideMex.cpp +├───toolbox/ +| ├───private/ +| | ├───powerMex.mexw64 +| | └───divideMex.mexw64 +| ├─... +| ├───powerNumber.m +| └───divideNumber.m +├───arithmetic.prj +└───buildfile.m +``` + + +```matlab +function plan = buildfile +% !!Revise to work in 24a!! + plan = buildplan(); + + mexOutputFolder = fullfile("toolbox","private"); + + % Compile all the folders inside cpp/*Mex into MEX functions + filesToMex = plan.files("cpp/mexfunction/*.cpp"); + for f = foldersToMex.paths + [~, fileName] = fileparts(f); + plan("mex:"+fileName) = matlab.buildtool.tasks.MexTask(fullfile(f, "**/*.cpp"), ... + mexOutputFolder); + end + plan("mex").Description = "Build MEX functions"; + +end +``` ## Incorporating External Libraries From 5206dc2ffbbfc28197b48d639a67ca413ac5872d Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 3 Jun 2025 16:41:32 +0530 Subject: [PATCH 075/105] Added multiple single source MEX functions section --- MEX.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index df1d002..9cc4bc5 100644 --- a/MEX.md +++ b/MEX.md @@ -146,6 +146,7 @@ arithmetic/ The `buildfile.m` is the same as before. ## Multiple single source MEX functions +For multiple C++ source files implementing its own MEX gateway, move the source file within the `mexfunctions` subfolder within the `cpp` folder. ``` text arithmetic/ @@ -163,7 +164,7 @@ arithmetic/ ├───arithmetic.prj └───buildfile.m ``` - +You can use the build file below to build all the MEX functions. ```matlab function plan = buildfile From d85e2ca5ea7a15e54a0cf97e26795ba0c73bfc85 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 3 Jun 2025 18:57:27 +0530 Subject: [PATCH 076/105] Checked the buildfile on 24b --- MEX.md | 35 +++++++++++++++++------------------ 1 file changed, 17 insertions(+), 18 deletions(-) diff --git a/MEX.md b/MEX.md index 9cc4bc5..5453262 100644 --- a/MEX.md +++ b/MEX.md @@ -79,21 +79,21 @@ Running the `mex` command for each MEX function can be tedious and error prone. ```matlab function plan = buildfile -% !!Revise to work in 24a!! Make sure it supports multiple *Mex directories and multiple .cpp files + % !!Revise to work in 24a!! + % plan = buildplan(); - + mexOutputFolder = fullfile("toolbox","private"); - - % Compile all the folders inside cpp/*Mex into MEX functions - foldersToMex = plan.files("cpp/*Mex").select(@isfolder); - for f = foldersToMex.paths - [~, folderName] = fileparts(f); - plan("mex:"+folderName) = matlab.buildtool.tasks.MexTask(fullfile(f, "**/*.cpp"), ... - mexOutputFolder, ... - Filename=folderName); + + % Compile Cpp source code within cpp/*Mex into MEX functions + foldersToMex = plan.files(fullfile("cpp", "*Mex")).select(@isfolder); + for folder = foldersToMex.paths + [~, folderName] = fileparts(folder); + plan("mex:"+folderName) = matlab.buildtool.tasks.MexTask(fullfile(folder, "**/*.cpp"), ... + mexOutputFolder, ... + Filename=folderName); end plan("mex").Description = "Build MEX functions"; - end ``` @@ -169,19 +169,18 @@ You can use the build file below to build all the MEX functions. ```matlab function plan = buildfile % !!Revise to work in 24a!! +% Do not have 24a installed on my machine, tried is on 24b worked fine. plan = buildplan(); - mexOutputFolder = fullfile("toolbox","private"); - % Compile all the folders inside cpp/*Mex into MEX functions - filesToMex = plan.files("cpp/mexfunction/*.cpp"); - for f = foldersToMex.paths - [~, fileName] = fileparts(f); - plan("mex:"+fileName) = matlab.buildtool.tasks.MexTask(fullfile(f, "**/*.cpp"), ... + % Compile all the folders inside cpp/*Mex.cpp into MEX functions + filesToMex = plan.files(fullfile("cpp", "mexfunctions", "*.cpp")); + for cppFile = filesToMex.paths + [~, fileName] = fileparts(cppFile); + plan("mex:"+fileName) = matlab.buildtool.tasks.MexTask(cppFile, ... mexOutputFolder); end plan("mex").Description = "Build MEX functions"; - end ``` From 7c5433b18112196731881ecabd599c6b13aa9ae2 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 5 Jun 2025 14:57:43 +0530 Subject: [PATCH 077/105] Updated the build file to work on 24a. --- MEX.md | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/MEX.md b/MEX.md index 5453262..51d19eb 100644 --- a/MEX.md +++ b/MEX.md @@ -8,7 +8,7 @@ - [X] Review and revise overview. - [X] Review and revise single source file section. - [X] Review and revise buildtool section. -- [ ] Update `buildfile.m` in buildtool section +- [ ] Update `buildfile.m` in buildtool section to work with 24a(24a does not have task collection) - [X] Review and update multiple source file section - [ ] Review and update `mexfunction` folder scenario section - [ ] Review and update external libraries section @@ -79,20 +79,23 @@ Running the `mex` command for each MEX function can be tedious and error prone. ```matlab function plan = buildfile - % !!Revise to work in 24a!! - % + % !! Works in 24a, introduced a dummy task called mex and made all the mex compile task as dependency. Not a great way to make it work. 24a does not have task collection. !! plan = buildplan(); mexOutputFolder = fullfile("toolbox","private"); % Compile Cpp source code within cpp/*Mex into MEX functions foldersToMex = plan.files(fullfile("cpp", "*Mex")).select(@isfolder); + mexTasks = string([]); for folder = foldersToMex.paths [~, folderName] = fileparts(folder); - plan("mex:"+folderName) = matlab.buildtool.tasks.MexTask(fullfile(folder, "**/*.cpp"), ... + plan("mex_"+folderName) = matlab.buildtool.tasks.MexTask(fullfile(folder, "**/*.cpp"), ... mexOutputFolder, ... Filename=folderName); + mexTasks(end+1) = "mex_" + folderName; end + plan("mex") = matlab.buildtool.Task; + plan("mex").Dependencies = mexTasks; plan("mex").Description = "Build MEX functions"; end ``` @@ -145,8 +148,8 @@ arithmetic/ The `buildfile.m` is the same as before. -## Multiple single source MEX functions -For multiple C++ source files implementing its own MEX gateway, move the source file within the `mexfunctions` subfolder within the `cpp` folder. +## Creating multiple single source MEX functions +If you have multiple C++ source files with each files having its own MEX gateway, the above approach of placing each C++ source file in a separate folder can become cumbersome. In such scenarios we recommend moving the source files within the `mexfunctions` subfolder under the `cpp` folder. ``` text arithmetic/ @@ -164,22 +167,26 @@ arithmetic/ ├───arithmetic.prj └───buildfile.m ``` -You can use the build file below to build all the MEX functions. +The build file for building these MEX functions is slightly different. ```matlab function plan = buildfile % !!Revise to work in 24a!! -% Do not have 24a installed on my machine, tried is on 24b worked fine. +% No task group in 24a, need to introduce a dummy mex task and make the actual tasks as a dependency plan = buildplan(); mexOutputFolder = fullfile("toolbox","private"); % Compile all the folders inside cpp/*Mex.cpp into MEX functions + mexTasks = string([]); filesToMex = plan.files(fullfile("cpp", "mexfunctions", "*.cpp")); for cppFile = filesToMex.paths [~, fileName] = fileparts(cppFile); - plan("mex:"+fileName) = matlab.buildtool.tasks.MexTask(cppFile, ... + plan("mex_"+fileName) = matlab.buildtool.tasks.MexTask(cppFile, ... mexOutputFolder); + mexTasks(end+1) = "mex_" + fileName; end + plan("mex") = matlab.buildtool.Task; + plan("mex").Dependencies = mexTasks; plan("mex").Description = "Build MEX functions"; end ``` From 0b0a455f7bc37b719d07bd42676ae8d6758a9ac6 Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Fri, 6 Jun 2025 08:42:51 -0400 Subject: [PATCH 078/105] Updated mexfunction scenario --- MEX.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/MEX.md b/MEX.md index 51d19eb..4a10faf 100644 --- a/MEX.md +++ b/MEX.md @@ -10,7 +10,7 @@ - [X] Review and revise buildtool section. - [ ] Update `buildfile.m` in buildtool section to work with 24a(24a does not have task collection) - [X] Review and update multiple source file section -- [ ] Review and update `mexfunction` folder scenario section +- [X] Review and update `mexfunction` folder scenario section - [ ] Review and update external libraries section - [ ] Review and update CI / GitHub Actions section @@ -117,7 +117,7 @@ arithmetic/ └───arithmetic.prj ``` -## Creating a MEX function from multiple C++ source files +## MEX function from multiple C++ source files Tha above pattern naturally extends to MEX functions that has multiple C++ source files. One of the source files must contain the [gateway](https://www.mathworks.com/help/matlab/matlab_external/gateway-routine.html) function. Place a the source files under a single folder. @@ -148,8 +148,8 @@ arithmetic/ The `buildfile.m` is the same as before. -## Creating multiple single source MEX functions -If you have multiple C++ source files with each files having its own MEX gateway, the above approach of placing each C++ source file in a separate folder can become cumbersome. In such scenarios we recommend moving the source files within the `mexfunctions` subfolder under the `cpp` folder. +## Handling a large number of MEX functions +If you have many MEX functions, each in its own C++ source file, the approach of placing each C++ source file in a separate folder is cumbersome. In such scenarios we recommend an alternate pattern: move the source files within a `mexfunctions` subfolder under the `cpp` folder. ``` text arithmetic/ From 3ad187f3898badb4d6319d02d98f191cbb9f52cc Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Fri, 6 Jun 2025 08:57:08 -0400 Subject: [PATCH 079/105] Review and update external libraries section --- MEX.md | 28 +++++++++------------------- 1 file changed, 9 insertions(+), 19 deletions(-) diff --git a/MEX.md b/MEX.md index 4a10faf..0cfd638 100644 --- a/MEX.md +++ b/MEX.md @@ -11,7 +11,7 @@ - [ ] Update `buildfile.m` in buildtool section to work with 24a(24a does not have task collection) - [X] Review and update multiple source file section - [X] Review and update `mexfunction` folder scenario section -- [ ] Review and update external libraries section +- [X] Review and update external libraries section - [ ] Review and update CI / GitHub Actions section ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) @@ -198,28 +198,19 @@ end -You can call libraries implemented in C++ using MEX functions. Since MEX source files are just C++ source code, they use the syntax of C++ to access external libraries. You may be wondering where to store these external libraries? +You can call libraries implemented in C++ using MEX functions. Since MEX source files are just C++ source code, they use standard C++ syntax to access external libraries. Where do we put these external libraries? -The answer to this question depends on the external library and the operating system with which you are working with. There are two broad categories of libraries: compile time and execution time libraries. - -We assume that the external library is available to you as include headers and binaries. We will not dwell into the details of how these headers and binaries are created. Let as talk a bit about it organization. +### External Library Header Files (`.h`,`.hpp`) +Create an `include` folder within the `cpp` folder and put the external library [header files](https://www.learncpp.com/cpp-tutorial/header-files/) within this folder. Use the [`-I`](https://www.mathworks.com/help/matlab/ref/mex.html#btw17rw-1-option1optionN) argument to the [mex function](https://www.mathworks.com/help/matlab/ref/mex.html) to specify that header files are in the `include` folder. -### External library headers -These files are required only at compile time, your users do not want them to run the MEX functions. Having a standard location to store these headers makes it easier for you to manage them and pass it to the compiler. - -Create an `include` folder within the `cpp` folder and move the external library headers within this folder. You can use [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) APIs [optional argument](https://www.mathworks.com/help/matlab/ref/mex.html#btw17rw-1-option1optionN) to ask the compiler to look for header files within the `include` folder. - -If you want to include a header only library, copy the library headers into the `include` folder within the `cpp` folder. - -### Interfacing with a compile time library -Toolbox users do not need these libraries for running the MEX functions, they are required only at compile time, these libraries are often referred to as static libraries. You can place these binaries under platform specific folders within the `library` folder. We recommend using standard names for the platform folders as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. The table below provides a summary of the folder names and file extensions used for static libraries for popular operating systems. +### Incorporating a Static Library (`.lib`) +Some MEX functions incorporate [static libraries](https://www.learncpp.com/cpp-tutorial/a1-static-and-dynamic-libraries/) that are compiled into your MEX function. Place these binaries under platform specific folders within the `library` folder. Use the names from the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB for the folder names. The table below provides a summary of the folder names and file extensions used for static libraries for popular operating systems. | Platform | Folder name | Binary Extension | | :---------------- | :------ | :------ | | Linux | glnxa64 | .a | | Windows | win64 | .lib | -| ARM Mac | maca64 | .dylib | -| Intel Mac | maci64 | .dylib | +| ARM / Intel Mac | maca64 | .dylib | ``` text zlibStatic/ @@ -247,9 +238,8 @@ zlibStatic/ └───buildfile.m ``` - -### Execution time libraries -These type of libraries are often referred to as shared object libraries or dynamic link libraries. These libraries are required for running the MEX functions and need to be shipped to the users. You can place the execution time binaries within the `private` folder under the `toolbox` folder, this makes sure that the library gets shipped to the user. You can use the -L and -l flags during compile time to specify the location and the name of the runtime library. +### Calling a Dynamic Library +[Dynamic libraries](https://www.learncpp.com/cpp-tutorial/a1-static-and-dynamic-libraries/) are required for running the MEX functions and must ship to the users. You can place the binaries within the `private` folder under the `toolbox` folder to ensure the library gets shipped to the user. You can use the [`-L`](https://www.mathworks.com/help/matlab/ref/mex.html#btw17rw-1-option1optionN) argument to the [mex function](https://www.mathworks.com/help/matlab/ref/mex.html) to specify the location and the name of the runtime library. ``` text From aa2cc2683e3fda89869b125bcc121de5463f3c79 Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Fri, 6 Jun 2025 09:01:52 -0400 Subject: [PATCH 080/105] added note on static vs dynamic --- MEX.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/MEX.md b/MEX.md index 0cfd638..3ce253b 100644 --- a/MEX.md +++ b/MEX.md @@ -241,6 +241,8 @@ zlibStatic/ ### Calling a Dynamic Library [Dynamic libraries](https://www.learncpp.com/cpp-tutorial/a1-static-and-dynamic-libraries/) are required for running the MEX functions and must ship to the users. You can place the binaries within the `private` folder under the `toolbox` folder to ensure the library gets shipped to the user. You can use the [`-L`](https://www.mathworks.com/help/matlab/ref/mex.html#btw17rw-1-option1optionN) argument to the [mex function](https://www.mathworks.com/help/matlab/ref/mex.html) to specify the location and the name of the runtime library. +**Note:** If you have a choice between using a static or dynamic library with your MEX function, we recommend using a static library. Static libraries are incorprorated inside your MEX function, making your MEX function more robust and reliable. + ``` text zlibShared/ @@ -269,11 +271,10 @@ loader's search path for different platforms. | Platform | Environment variable for loader's search path | | :-------- | :-------------------------------------------- | -| Linux | LD_LIBRARY_PATH | -| Windows | PATH | -| Mac | DYLD_LIBRARY_PATH | +| Linux | `LD_LIBRARY_PATH` | +| Windows | `PATH` | +| Mac | `DYLD_LIBRARY_PATH` | -For non C++ libraries, you need to start MATLAB from an environment where the loaders's search path is already established. From fe51fccfe9d168a724a38fe9db2a3c0b2625410d Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Fri, 6 Jun 2025 18:35:07 +0530 Subject: [PATCH 081/105] Minor change --- MEX.md | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/MEX.md b/MEX.md index 51d19eb..d68870e 100644 --- a/MEX.md +++ b/MEX.md @@ -10,9 +10,10 @@ - [X] Review and revise buildtool section. - [ ] Update `buildfile.m` in buildtool section to work with 24a(24a does not have task collection) - [X] Review and update multiple source file section -- [ ] Review and update `mexfunction` folder scenario section +- [ ] Review and update `mexfunction` folder scenario section (updated the section, added the folder structure and build file) - [ ] Review and update external libraries section - [ ] Review and update CI / GitHub Actions section +- [ ] External: GitHub repos for examples ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) @@ -101,7 +102,7 @@ end ``` -Our example toolbox adds a `buildfile.m` to automate the building of the MEX functions: +Our example toolbox adds a `buildfile.m` to automate the building of the MEX functions, on executing `buildtool mex`, the `invertMex.mexw64` gets created within the `private` folder. ``` text arithmetic/ @@ -119,7 +120,7 @@ arithmetic/ ## Creating a MEX function from multiple C++ source files -Tha above pattern naturally extends to MEX functions that has multiple C++ source files. One of the source files must contain the [gateway](https://www.mathworks.com/help/matlab/matlab_external/gateway-routine.html) function. Place a the source files under a single folder. +Tha above pattern naturally extends to MEX functions that has multiple C++ source files. One of the source files must contain the [gateway](https://www.mathworks.com/help/matlab/matlab_external/gateway-routine.html) function. Place the source files under a single folder. -You can call libraries implemented in C++ using MEX functions. Since MEX source files are just C++ source code, they use the syntax of C++ to access external libraries. You may be wondering where to store these external libraries? +If you have a MEX function that make a call to a an external C++ library and you might be wondering how to organize these external libraries? The answer to this question depends on the external library and the operating system with which you are working with. There are two broad categories of libraries: compile time and execution time libraries. -We assume that the external library is available to you as include headers and binaries. We will not dwell into the details of how these headers and binaries are created. Let as talk a bit about it organization. +We assume that the external library is available to you as include headers and binaries. We will not dwell into the details of how these headers and binaries are created. Let as talk a bit about its organization. ### External library headers These files are required only at compile time, your users do not want them to run the MEX functions. Having a standard location to store these headers makes it easier for you to manage them and pass it to the compiler. @@ -212,7 +213,7 @@ Create an `include` folder within the `cpp` folder and move the external library If you want to include a header only library, copy the library headers into the `include` folder within the `cpp` folder. ### Interfacing with a compile time library -Toolbox users do not need these libraries for running the MEX functions, they are required only at compile time, these libraries are often referred to as static libraries. You can place these binaries under platform specific folders within the `library` folder. We recommend using standard names for the platform folders as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. The table below provides a summary of the folder names and file extensions used for static libraries for popular operating systems. +Like library headers, these libraries are not needed for running the MEX functions, they are required only at compile time, these libraries are often referred to as static libraries. You can place these binaries under platform specific folders within the `library` folder. We recommend using standard names for the platform folders as defined by the [`computer('arch')`](https://www.mathworks.com/help/matlab/ref/computer.html) command in MATLAB. The table below provides a summary of the folder names and file extensions used for static libraries for popular operating systems. | Platform | Folder name | Binary Extension | | :---------------- | :------ | :------ | @@ -249,7 +250,7 @@ zlibStatic/ ### Execution time libraries -These type of libraries are often referred to as shared object libraries or dynamic link libraries. These libraries are required for running the MEX functions and need to be shipped to the users. You can place the execution time binaries within the `private` folder under the `toolbox` folder, this makes sure that the library gets shipped to the user. You can use the -L and -l flags during compile time to specify the location and the name of the runtime library. +These type of libraries are often referred to as shared object libraries or dynamic link libraries. These libraries are required for running the MEX functions and need to be shipped to the users. You can place the execution time binaries within the `private` folder under the `toolbox` folder, this makes sure that the library gets shipped to the user. Make sure to use `-L` and `-l` flags during compile time to specify the location and the name of the runtime library. ``` text @@ -281,7 +282,8 @@ loader's search path for different platforms. | :-------- | :-------------------------------------------- | | Linux | LD_LIBRARY_PATH | | Windows | PATH | -| Mac | DYLD_LIBRARY_PATH | +| Mac | DYLD_LIBRARY_PATH | + For non C++ libraries, you need to start MATLAB from an environment where the loaders's search path is already established. From 0b4476ce8a2444d444bf1d7a78de6e2118f65d82 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Fri, 6 Jun 2025 19:01:46 +0530 Subject: [PATCH 082/105] added tl;dr section --- MEX.md | 32 +++++++++++++++++++++----------- 1 file changed, 21 insertions(+), 11 deletions(-) diff --git a/MEX.md b/MEX.md index 6c33185..e9ab6df 100644 --- a/MEX.md +++ b/MEX.md @@ -8,17 +8,26 @@ - [X] Review and revise overview. - [X] Review and revise single source file section. - [X] Review and revise buildtool section. -- [ ] Update `buildfile.m` in buildtool section to work with 24a(24a does not have task collection) +- [ ] Update `buildfile.m` in buildtool section to work with 24b - [X] Review and update multiple source file section - [X] Review and update `mexfunction` folder scenario section - [X] Review and update external libraries section - [ ] Review and update CI / GitHub Actions section -- [ ] External: GitHub repos for examples +- [ ] Extend Arithmetic to build on all three OS- single MEX API +- [ ] Appendix for 2024a buildfiles ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) Welcome to the MATLAB® MEX Best Practice guide, which extends [MATLAB Toolbox Best Practices](./README.md) focusing on integrating [MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) into your MATLAB toolbox. MEX functions enable you to harness the power of C, C++, and Fortran code within MATLAB. In this document when we say "C++", we mean "C, C++, and Fortran." +## TL;DR +- C++ code goes into the `cpp` folder. +- Each MEX functon is in its own folder with a `Mex` suffix +- Place the built MEX functions under a `private` folder under your `toolbox` folder. MEX functions should only be called from within your toolbox in order to increase reliablity. +- External libraries are placed within a platform specific folder +- We recommend using the `mexhost` command to increase relaiablity +- Use a MexTask in your buildfile.m + ## Overview [MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) are compiled functions that bridge the gap between MATLAB and C++. They behave like a MATLAB function and you must build them for each operating system you want to run on. You can determine the MEX file extension (for example, `.mexw64` in Microsoft Windows) for your operating system using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). This guide will navigate you through the process of integrating of MEX functions, ensuring smooth implementation in both development and production environments. This makes it easier for others to understand and contribute to your toolbox. @@ -311,9 +320,13 @@ arithmetic/ ``` --> -## Multi platform MEX functions build using CI systems -## Testing +## Automating Builds with GitHub Actions + +### Multi platform MEX functions build using CI systems +**TBD** + + ## Conclusion @@ -358,6 +365,9 @@ By following to these best practices, you'll create a robust, maintainable, and We welcome your input! For further details, suggestions, or to contribute, please [open an issue](https://github.com/mathworks/toolboxdesign/issues/new/choose). +## Appendeix + + --- [![CC-BY-4.0](images/cc-by-40.png)](https://creativecommons.org/licenses/by/4.0/) From 4e304046a63841606bf80a1b6d855c7cdd6da3c0 Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Fri, 6 Jun 2025 15:02:00 -0400 Subject: [PATCH 083/105] improved the language and made it more consistent. --- MEX.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/MEX.md b/MEX.md index e9ab6df..0ff36a3 100644 --- a/MEX.md +++ b/MEX.md @@ -18,24 +18,24 @@ ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) -Welcome to the MATLAB® MEX Best Practice guide, which extends [MATLAB Toolbox Best Practices](./README.md) focusing on integrating [MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) into your MATLAB toolbox. MEX functions enable you to harness the power of C, C++, and Fortran code within MATLAB. In this document when we say "C++", we mean "C, C++, and Fortran." +Welcome to the MATLAB® MEX Best Practice guide, which extends [MATLAB Toolbox Best Practices](./README.md). This document focuses on integrating [MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) into your MATLAB toolbox. MEX functions enable you to harness the power of C, C++, and Fortran code within MATLAB. In this document when we say "C++", we mean "C, C++, and Fortran." ## TL;DR - C++ code goes into the `cpp` folder. - Each MEX functon is in its own folder with a `Mex` suffix -- Place the built MEX functions under a `private` folder under your `toolbox` folder. MEX functions should only be called from within your toolbox in order to increase reliablity. +- Place the built MEX functions in a `private` folder inside your `toolbox` folder. MEX functions should only be called from within your toolbox in order to increase reliablity. - External libraries are placed within a platform specific folder - We recommend using the `mexhost` command to increase relaiablity -- Use a MexTask in your buildfile.m +- Use a [MexTask](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) in your [buildfile.m](https://www.mathworks.com/help/matlab/build-automation.html) for consistent builds. ## Overview [MEX functions](https://www.mathworks.com/help/matlab/call-mex-file-functions.html) are compiled functions that bridge the gap between MATLAB and C++. They behave like a MATLAB function and you must build them for each operating system you want to run on. You can determine the MEX file extension (for example, `.mexw64` in Microsoft Windows) for your operating system using [`mexext`](https://www.mathworks.com/help/matlab/ref/mexext.html). This guide will navigate you through the process of integrating of MEX functions, ensuring smooth implementation in both development and production environments. This makes it easier for others to understand and contribute to your toolbox. -To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this toolbox throughout the guide to demonstrate practical applications of these principles. For key concepts please refer to the [Toolbox Best Practices](./README.md). +To illustrate these best practices, we've created a sample project: The Arithmetic Toolbox, available on [GitHub](https://github.com/mathworks/arithmetic). We'll reference this toolbox throughout the guide to demonstrate practical applications of these principles. For key concepts, refer to the [Toolbox Best Practices](./README.md). ## MEX function from a single C++ source file -Suppose you have a single C++ MEX source file, this MEX source file need not be distributed to the toolbox users, since they are not required to run the toolbox. Only the compiled function needs to be distributed. We recommend keeping the C++ MEX source files outside of the `toolbox` folder in a folder focused on C++ code, `cpp`. +The most common case is when you have a single C++ MEX source file. You do not distribute the source file to toolbox users, only the compiled mex function. We recommend keeping the C++ MEX source files outside of the `toolbox` folder in a folder focused on C++ code, `cpp`. For each MEX function in the `cpp` folder, create a folder with the name that matches the name of your MEX function. This folder should end with `Mex` to indicate that all the files within the folder are associated with a single MEX function. @@ -52,7 +52,7 @@ arithmetic/ ``` ### Building MEX functions -You can use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command, to compile `invertMex.cpp` into MEX functions. We suggest you place your compiled MEX functions in your `private` folder within the `toolbox` folder (see below). You need to provide the path of `invertMex.cpp` and path of the `private` folder as inputs to the `mex` command. +Use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command to compile `invertMex.cpp` into MEX functions. Place your compiled MEX functions in a `private` folder within the `toolbox` folder (see below). You need to provide the path of `invertMex.cpp` and path of the `private` folder as inputs to the `mex` command. ```matlab >> source = fullfile("cpp", "invertMex", "*.cpp"); @@ -80,7 +80,7 @@ arithmetic/ ``` ### Additional Notes * **Why put the MEX functions within a private folder?** By putting it in a [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html) folder, you restrict your users from calling the MEX function directly. Even minor errors in a MEX function will crash MATLAB, especially if they receive unexpected inputs. By limiting access to MEX functions to a MATLAB function that you control, you ensure that only what you expect will be passed as input to the MEX function, preventing errors from unexpected or unhandled inputs. - * **Out of process MEX host** We recommend [Out-of-Process Execution of C++ MEX Functions](https://www.mathworks.com/help/matlab/matlab_external/out-of-process-execution-of-c-mex-functions.html). This isolates the MATLAB process from crashes in you C++ MEX function and allows you to use some third-party libraries that are not compatible with MATLAB. Use the [mexhost](https://www.mathworks.com/help/matlab/ref/mexhost.html) command to do this. Note that `mexhost` is only supported for C++ MEX functions. + * **Out of process MEX host** We recommend [Out-of-Process Execution of C++ MEX Functions](https://www.mathworks.com/help/matlab/matlab_external/out-of-process-execution-of-c-mex-functions.html). This prevents coding errors in your C++ MEX function from crashing MATLAB and allows you to use some third-party libraries that are not compatible with MATLAB. Use the [mexhost](https://www.mathworks.com/help/matlab/ref/mexhost.html) command. Note that `mexhost` is only supported for C++ MEX functions. * **Using git** In git source control systems, we recommend that you do not keep compiled MEX functions under version control, as they are derived files. Add `*.mex*` to your `.gitignore` file. This is part of the [standard .gitignore file](https://github.com/mathworks/gitignore/blob/main/Global/MATLAB.gitignore) for MATLAB. ### Automation using `buildtool` @@ -129,13 +129,13 @@ arithmetic/ ## MEX function from multiple C++ source files -Tha above pattern naturally extends to MEX functions that has multiple C++ source files. One of the source files must contain the [gateway](https://www.mathworks.com/help/matlab/matlab_external/gateway-routine.html) function. Place the source files under a single folder. +The above pattern extends to MEX functions that has multiple C++ source files. One of the source files must contain the [gateway](https://www.mathworks.com/help/matlab/matlab_external/gateway-routine.html) function. Place all the source files under a single folder. -Our example toolbox adds two .cpp files in the `subtractMex` folder, which builds to the `subtractMex.mexw64` binary in the `private` folder. `subtractNumber.m` provides a user accessible version that error checks: +Our example toolbox adds two .cpp files in the `subtractMex` folder, which builds to the `subtractMex.mexw64` binary in the `private` folder. `subtractNumber.m` provides a user accessible version that error checks before calling `SubtractMex`: ``` text arithmetic/ @@ -208,7 +208,7 @@ end -You can call libraries implemented in C++ using MEX functions. Since MEX source files are just C++ source code, they use standard C++ syntax to access external libraries. Where do we put these external libraries? +You can call libraries implemented in C++ using MEX functions. Since MEX source files are just C++ source code, they use standard C++ syntax to access external libraries. ### External Library Header Files (`.h`,`.hpp`) Create an `include` folder within the `cpp` folder and put the external library [header files](https://www.learncpp.com/cpp-tutorial/header-files/) within this folder. Use the [`-I`](https://www.mathworks.com/help/matlab/ref/mex.html#btw17rw-1-option1optionN) argument to the [mex function](https://www.mathworks.com/help/matlab/ref/mex.html) to specify that header files are in the `include` folder. @@ -249,7 +249,7 @@ zlibStatic/ ``` ### Calling a Dynamic Library -[Dynamic libraries](https://www.learncpp.com/cpp-tutorial/a1-static-and-dynamic-libraries/) are required for running the MEX functions and must ship to the users. You can place the binaries within the `private` folder under the `toolbox` folder to ensure the library gets shipped to the user. You can use the [`-L`](https://www.mathworks.com/help/matlab/ref/mex.html#btw17rw-1-option1optionN) argument to the [mex function](https://www.mathworks.com/help/matlab/ref/mex.html) to specify the location and the name of the runtime library. +[Dynamic libraries](https://www.learncpp.com/cpp-tutorial/a1-static-and-dynamic-libraries/) are required for running the MEX functions and must ship to the users. Place the binaries within the `private` folder under the `toolbox` folder to ensure the library gets shipped to the user. Use the [`-L`](https://www.mathworks.com/help/matlab/ref/mex.html#btw17rw-1-option1optionN) argument to the [mex function](https://www.mathworks.com/help/matlab/ref/mex.html) to specify the location and the name of the runtime library. **Note:** If you have a choice between using a static or dynamic library with your MEX function, we recommend using a static library. Static libraries are incorprorated inside your MEX function, making your MEX function more robust and reliable. @@ -276,7 +276,7 @@ zlibShared/ ``` * For projects with complex dependencies, consider adopting dependency management tools like [Conan](https://conan.io/) which can significantly simplify library management across different platforms. -* Depending on the platform, execution time libraries require their path to be added to the loader's search path. These search paths are often establish using platform dependent environment variables. In case of C++ libraries you can use [`mexhost`](https://www.mathworks.com/help/matlab/ref/mexhost.html)'s `EnvironmentVariables` option to set-up the loader's path. Here is a summary of +* Depending on the platform, dynamic libraries require adding their path to the operating system search path. These search paths are often set using environment variables. In case of C++ libraries you can use [`mexhost`](https://www.mathworks.com/help/matlab/ref/mexhost.html)'s `EnvironmentVariables` option to set-up the loader's path. Here is a summary of loader's search path for different platforms. | Platform | Environment variable for loader's search path | @@ -361,7 +361,7 @@ arithmetic/ ## Conclusion -By following to these best practices, you'll create a robust, maintainable, and user-friendly MATLAB toolbox that harnesses the full potential of MEX functions. Through effective project structure organization, build automation, and dependency management, you can focus on delivering great solutions to your users. +By following to these best practices, you'll create a reliable, maintainable, and user-friendly MATLAB toolbox that harnesses the full potential of MEX functions. Through effective project structure organization, build automation, and dependency management, you can focus on delivering great solutions to your users. We welcome your input! For further details, suggestions, or to contribute, please [open an issue](https://github.com/mathworks/toolboxdesign/issues/new/choose). From 443501a130f8c2e5cfc7a91b756550695a9f1388 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 12 Jun 2025 00:41:35 +0530 Subject: [PATCH 084/105] Added an appendix --- MEX.md | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index 0ff36a3..f940e93 100644 --- a/MEX.md +++ b/MEX.md @@ -322,7 +322,9 @@ arithmetic/ ## Automating Builds with GitHub Actions +For setting up GitHub Runner refer to [this page](https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners). +MAC on GH Actions take a lot of time to execute. ### Multi platform MEX functions build using CI systems **TBD** @@ -366,7 +368,26 @@ By following to these best practices, you'll create a reliable, maintainable, an We welcome your input! For further details, suggestions, or to contribute, please [open an issue](https://github.com/mathworks/toolboxdesign/issues/new/choose). ## Appendeix - +MATLAB 2024b introduced task collection, it makes managing multiple mex tasks much easier. +```matlab +function plan = buildfile + % Works in 24b. + + plan = buildplan(); + + mexOutputFolder = fullfile("toolbox","private"); + + % Compile Cpp source code within cpp/*Mex into MEX functions + foldersToMex = plan.files(fullfile("cpp", "*Mex")).select(@isfolder); + for folder = foldersToMex.paths + [~, folderName] = fileparts(folder); + plan("mex:"+folderName) = matlab.buildtool.tasks.MexTask(fullfile(folder, "**/*.cpp"), ... + mexOutputFolder, ... + Filename=folderName); + end + plan("mex").Description = "Build MEX functions"; +end +``` --- [![CC-BY-4.0](images/cc-by-40.png)](https://creativecommons.org/licenses/by/4.0/) From 07860de088bbe9baee28ab79aec108aedc63deb1 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 12 Jun 2025 16:13:42 +0530 Subject: [PATCH 085/105] Added an introduction to GitHub Actions --- MEX.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index f940e93..25c6560 100644 --- a/MEX.md +++ b/MEX.md @@ -322,6 +322,9 @@ arithmetic/ ## Automating Builds with GitHub Actions +You can use [MATLAB Actions](https://github.com/matlab-actions) for building, testing and deploying MATLAB toolboxes. MathWorks offers free MATLAB licenses for configuring MATLAB within a GitHub Actions for public GitHub repositories. If you GitHub repository is private you will need a [batch license token](https://github.com/mathworks-ref-arch/matlab-dockerfile/blob/main/alternates/non-interactive/MATLAB-BATCH.md#matlab-batch-licensing-token) to run MATLAB on [GitHub Hosted Runners](https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners). + + For setting up GitHub Runner refer to [this page](https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners). MAC on GH Actions take a lot of time to execute. @@ -372,7 +375,7 @@ MATLAB 2024b introduced task collection, it makes managing multiple mex tasks mu ```matlab function plan = buildfile % Works in 24b. - + plan = buildplan(); mexOutputFolder = fullfile("toolbox","private"); From 3f43757778bc5cb68fcabfbed90fa6c41c3ff5f0 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 12 Jun 2025 16:53:50 +0530 Subject: [PATCH 086/105] Added the first section for GitHub Actions --- MEX.md | 42 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 36 insertions(+), 6 deletions(-) diff --git a/MEX.md b/MEX.md index 25c6560..50b0c6d 100644 --- a/MEX.md +++ b/MEX.md @@ -322,14 +322,44 @@ arithmetic/ ## Automating Builds with GitHub Actions -You can use [MATLAB Actions](https://github.com/matlab-actions) for building, testing and deploying MATLAB toolboxes. MathWorks offers free MATLAB licenses for configuring MATLAB within a GitHub Actions for public GitHub repositories. If you GitHub repository is private you will need a [batch license token](https://github.com/mathworks-ref-arch/matlab-dockerfile/blob/main/alternates/non-interactive/MATLAB-BATCH.md#matlab-batch-licensing-token) to run MATLAB on [GitHub Hosted Runners](https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners). - - -For setting up GitHub Runner refer to [this page](https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners). +You can use [MATLAB Actions](https://github.com/matlab-actions) for configuring MATLAB within a [GitHub Action](https://docs.github.com/en/actions). It can be used to build, test and deploy your toolbox. MathWorks offers free licenses for configuring MATLAB within a GitHub Action for public GitHub repositories. If your GitHub repository is private, you will need a [batch license token](https://github.com/mathworks-ref-arch/matlab-dockerfile/blob/main/alternates/non-interactive/MATLAB-BATCH.md#matlab-batch-licensing-token) to run MATLAB on [GitHub Runners](https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners). GitHub Hosted Runners offer support for all the three major operating systems Windows, Mac and Linux. + +Within a GitHub Action, you can invoke MATLAB's buildtool using [matlab-actions/run-command@v2](https://github.com/matlab-actions/run-command/). The build tasks that you already configured for local development like building MEX functions, running tests and packaging the toolbox can all be reused within your GitHub Workflow. + +```yml +name: releaseWorkflow + +# This workflows creates a new (draft) release when a new git tag is pushed to GitHub. +on: + push: + tags: + - '*' + workflow_dispatch: + +jobs: + buildPackageAndRelease: + runs-on: windows-latest + + steps: + - name: Check out repository + uses: actions/checkout@v4 + - name: Set up MATLAB + uses: matlab-actions/setup-matlab@v2 + with: + release: R2024b + - name: Run script + uses: matlab-actions/run-command@v2 + with: + command: buildtool mex test release + - name: Create GitHub Release + uses: ncipollo/release-action@v1 + with: + draft: true + artifacts: "release/Arithmetic_Toolbox.mltbx" +``` -MAC on GH Actions take a lot of time to execute. ### Multi platform MEX functions build using CI systems -**TBD** +MAC on GH Actions take a lot of time to execute. Our example toolbox adds a `buildfile.m` to automate the building of the MEX functions, on executing `buildtool mex`, the `invertMex.mexw64` gets created within the `private` folder. @@ -181,22 +179,17 @@ The build file for building these MEX functions is slightly different. ```matlab function plan = buildfile -% !!Revise to work in 24a!! -% No task group in 24a, need to introduce a dummy mex task and make the actual tasks as a dependency +% !!Revise to work in 24b!! plan = buildplan(); mexOutputFolder = fullfile("toolbox","private"); % Compile all the folders inside cpp/*Mex.cpp into MEX functions - mexTasks = string([]); filesToMex = plan.files(fullfile("cpp", "mexfunctions", "*.cpp")); for cppFile = filesToMex.paths [~, fileName] = fileparts(cppFile); - plan("mex_"+fileName) = matlab.buildtool.tasks.MexTask(cppFile, ... + plan("mex:"+fileName) = matlab.buildtool.tasks.MexTask(cppFile, ... mexOutputFolder); - mexTasks(end+1) = "mex_" + fileName; end - plan("mex") = matlab.buildtool.Task; - plan("mex").Dependencies = mexTasks; plan("mex").Description = "Build MEX functions"; end ``` @@ -350,7 +343,7 @@ jobs: - name: Run script uses: matlab-actions/run-command@v2 with: - command: buildtool mex test release + command: buildtool mex test package - name: Create GitHub Release uses: ncipollo/release-action@v1 with: @@ -417,7 +410,7 @@ jobs: - name: Run script uses: matlab-actions/run-command@v2 with: - command: buildtool release, ls release + command: buildtool test package - name: Create GitHub Release uses: ncipollo/release-action@v1 with: @@ -464,24 +457,50 @@ By following to these best practices, you'll create a reliable, maintainable, an We welcome your input! For further details, suggestions, or to contribute, please [open an issue](https://github.com/mathworks/toolboxdesign/issues/new/choose). -## Appendeix -MATLAB 2024b introduced task collection, it makes managing multiple mex tasks much easier. +## Appendix: Buildfiles for MATLAB R2024a + ```matlab function plan = buildfile - % Works in 24b. - + % !! Works in 24a, introduced a dummy task called mex and made all the mex compile task as dependency. Not a great way to make it work. 24a does not have task collection. !! plan = buildplan(); mexOutputFolder = fullfile("toolbox","private"); % Compile Cpp source code within cpp/*Mex into MEX functions foldersToMex = plan.files(fullfile("cpp", "*Mex")).select(@isfolder); + mexTasks = string([]); for folder = foldersToMex.paths [~, folderName] = fileparts(folder); - plan("mex:"+folderName) = matlab.buildtool.tasks.MexTask(fullfile(folder, "**/*.cpp"), ... + plan("mex_"+folderName) = matlab.buildtool.tasks.MexTask(fullfile(folder, "**/*.cpp"), ... mexOutputFolder, ... Filename=folderName); + mexTasks(end+1) = "mex_" + folderName; end + plan("mex") = matlab.buildtool.Task; + plan("mex").Dependencies = mexTasks; + plan("mex").Description = "Build MEX functions"; +end +``` + + +```matlab +function plan = buildfile +% !!Revise to work in 24a!! +% No task group in 24a, need to introduce a dummy mex task and make the actual tasks as a dependency + plan = buildplan(); + mexOutputFolder = fullfile("toolbox","private"); + + % Compile all the folders inside cpp/*Mex.cpp into MEX functions + mexTasks = string([]); + filesToMex = plan.files(fullfile("cpp", "mexfunctions", "*.cpp")); + for cppFile = filesToMex.paths + [~, fileName] = fileparts(cppFile); + plan("mex_"+fileName) = matlab.buildtool.tasks.MexTask(cppFile, ... + mexOutputFolder); + mexTasks(end+1) = "mex_" + fileName; + end + plan("mex") = matlab.buildtool.Task; + plan("mex").Dependencies = mexTasks; plan("mex").Description = "Build MEX functions"; end ``` From 70700b398d86215a9e6fd132abbb0b89a24f35d6 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 12 Jun 2025 19:00:44 +0530 Subject: [PATCH 089/105] Minor updates --- MEX.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/MEX.md b/MEX.md index 2949ad4..170d009 100644 --- a/MEX.md +++ b/MEX.md @@ -23,10 +23,10 @@ Welcome to the MATLAB® MEX Best Practice guide, which extends [MATLAB Toolbo ## TL;DR - C++ code goes into the `cpp` folder. - Each MEX functon is in its own folder with a `Mex` suffix -- Place the built MEX functions in a `private` folder inside your `toolbox` folder. MEX functions should only be called from within your toolbox in order to increase reliablity. +- Place the built MEX functions in a `private` folder inside your `toolbox` folder. MEX functions should only be called from within your toolbox in order to increase reliablity - External libraries are placed within a platform specific folder - We recommend using the `mexhost` command to increase relaiablity -- Use a [MexTask](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) in your [buildfile.m](https://www.mathworks.com/help/matlab/build-automation.html) for consistent builds. +- Use a [MexTask](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) in your [buildfile.m](https://www.mathworks.com/help/matlab/build-automation.html) for consistent builds ## Overview From 9ab9bd8f39a13ca2ceb9df50d7cd120ead2f0743 Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Thu, 12 Jun 2025 09:34:27 -0400 Subject: [PATCH 090/105] Tweaks to the TL;DR and syntax stuff --- MEX.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/MEX.md b/MEX.md index 2949ad4..42ae8e7 100644 --- a/MEX.md +++ b/MEX.md @@ -24,8 +24,8 @@ Welcome to the MATLAB® MEX Best Practice guide, which extends [MATLAB Toolbo - C++ code goes into the `cpp` folder. - Each MEX functon is in its own folder with a `Mex` suffix - Place the built MEX functions in a `private` folder inside your `toolbox` folder. MEX functions should only be called from within your toolbox in order to increase reliablity. -- External libraries are placed within a platform specific folder -- We recommend using the `mexhost` command to increase relaiablity +- External libraries are placed within a platform specific folder in the `private` folder and added to the system path +- We recommend using the [`mexhost`](https://www.mathworks.com/help/matlab/ref/mexhost.html) command to increase reliablity - Use a [MexTask](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) in your [buildfile.m](https://www.mathworks.com/help/matlab/build-automation.html) for consistent builds. ## Overview @@ -79,8 +79,8 @@ arithmetic/ └───arithmetic.prj ``` ### Additional Notes - * **Why put the MEX functions within a private folder?** By putting it in a [private](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html) folder, you restrict your users from calling the MEX function directly. Even minor errors in a MEX function will crash MATLAB, especially if they receive unexpected inputs. By limiting access to MEX functions to a MATLAB function that you control, you ensure that only what you expect will be passed as input to the MEX function, preventing errors from unexpected or unhandled inputs. - * **Out of process MEX host** We recommend [Out-of-Process Execution of C++ MEX Functions](https://www.mathworks.com/help/matlab/matlab_external/out-of-process-execution-of-c-mex-functions.html). This prevents coding errors in your C++ MEX function from crashing MATLAB and allows you to use some third-party libraries that are not compatible with MATLAB. Use the [mexhost](https://www.mathworks.com/help/matlab/ref/mexhost.html) command. Note that `mexhost` is only supported for C++ MEX functions. + * **Why put the MEX functions within a private folder?** By putting it in a [`private`](https://www.mathworks.com/help/matlab/matlab_prog/private-functions.html) folder, you restrict your users from calling the MEX function directly. Even minor errors in a MEX function will crash MATLAB, especially if they receive unexpected inputs. By limiting access to MEX functions to a MATLAB function that you control, you ensure that only what you expect will be passed as input to the MEX function, preventing errors from unexpected or unhandled inputs. + * **Out of process MEX host** We recommend [Out-of-Process Execution of C++ MEX Functions](https://www.mathworks.com/help/matlab/matlab_external/out-of-process-execution-of-c-mex-functions.html). This prevents coding errors in your C++ MEX function from crashing MATLAB and allows you to use some third-party libraries that are not compatible with MATLAB. Use the [`mexhost`](https://www.mathworks.com/help/matlab/ref/mexhost.html) command. Note that `mexhost` is only supported for C++ MEX functions. * **Using git** In git source control systems, we recommend that you do not keep compiled MEX functions under version control, as they are derived files. Add `*.mex*` to your `.gitignore` file. This is part of the [standard .gitignore file](https://github.com/mathworks/gitignore/blob/main/Global/MATLAB.gitignore) for MATLAB. ### Automation using `buildtool` From 4c743e0175615a9268be08523e8282aaa6be1102 Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Thu, 12 Jun 2025 09:39:03 -0400 Subject: [PATCH 091/105] update todo --- MEX.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index b8e2064..f132e58 100644 --- a/MEX.md +++ b/MEX.md @@ -14,7 +14,7 @@ - [X] Review and update external libraries section - [ ] Review and update CI / GitHub Actions section - [ ] Extend Arithmetic to build on all three OS- single MEX API -- [ ] Appendix for 2024a buildfiles +- [ ] Appendix for 2024a and earlier buildfile that just uses `mex` ![Version Number](https://img.shields.io/github/v/release/mathworks/toolboxdesign?label=version) ![CC-BY-4.0 License](https://img.shields.io/github/license/mathworks/toolboxdesign) From 9da54829d6e83a2c54ea7c4a7dad4662e4c7829c Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 12 Jun 2025 19:47:58 +0530 Subject: [PATCH 092/105] Updated the GitHub Actions section --- MEX.md | 110 ++++++++++----------------------------------------------- 1 file changed, 18 insertions(+), 92 deletions(-) diff --git a/MEX.md b/MEX.md index b8e2064..129fdef 100644 --- a/MEX.md +++ b/MEX.md @@ -242,7 +242,7 @@ zlibStatic/ ``` ### Calling a Dynamic Library -[Dynamic libraries](https://www.learncpp.com/cpp-tutorial/a1-static-and-dynamic-libraries/) are required for running the MEX functions and must ship to the users. Place the binaries within the `private` folder under the `toolbox` folder to ensure the library gets shipped to the user. Use the [`-L`](https://www.mathworks.com/help/matlab/ref/mex.html#btw17rw-1-option1optionN) argument to the [mex function](https://www.mathworks.com/help/matlab/ref/mex.html) to specify the location and the name of the runtime library. +[Dynamic libraries](https://www.learncpp.com/cpp-tutorial/a1-static-and-dynamic-libraries/) are required for running the MEX functions and must ship to the users. Place the binaries within the `private` folder under the `toolbox` folder to ensure the library gets shipped to the user. Use the [`-L`](https://www.mathworks.com/help/matlab/ref/mex.html#btw17rw-1-option1optionN) argument to the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command to specify the location and the name of the runtime library. **Note:** If you have a choice between using a static or dynamic library with your MEX function, we recommend using a static library. Static libraries are incorprorated inside your MEX function, making your MEX function more robust and reliable. @@ -269,10 +269,9 @@ zlibShared/ ``` * For projects with complex dependencies, consider adopting dependency management tools like [Conan](https://conan.io/) which can significantly simplify library management across different platforms. -* Depending on the platform, dynamic libraries require adding their path to the operating system search path. These search paths are often set using environment variables. In case of C++ libraries you can use [`mexhost`](https://www.mathworks.com/help/matlab/ref/mexhost.html)'s `EnvironmentVariables` option to set-up the loader's path. Here is a summary of -loader's search path for different platforms. +* Depending on the platform, dynamic libraries require adding their path to the operating system search path. These search paths are often set using environment variables. In case of C++ libraries you can use [`mexhost`](https://www.mathworks.com/help/matlab/ref/mexhost.html)'s `EnvironmentVariables` option to set-up the loader's path. The table below shows the environment variable for different operating systems. -| Platform | Environment variable for loader's search path | +| Operating system | Environment variable for loader's search path | | :-------- | :-------------------------------------------- | | Linux | `LD_LIBRARY_PATH` | | Windows | `PATH` | @@ -315,108 +314,35 @@ arithmetic/ ## Automating Builds with GitHub Actions -You can use [MATLAB Actions](https://github.com/matlab-actions) for configuring MATLAB within a [GitHub Action](https://docs.github.com/en/actions). It can be used to build, test and deploy your toolbox. MathWorks offers free licenses for configuring MATLAB within a GitHub Action for public GitHub repositories. If your GitHub repository is private, you will need a [batch license token](https://github.com/mathworks-ref-arch/matlab-dockerfile/blob/main/alternates/non-interactive/MATLAB-BATCH.md#matlab-batch-licensing-token) to run MATLAB on [GitHub Runners](https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners). GitHub Hosted Runners offer support for all the three major operating systems Windows, Mac and Linux. +[MATLAB Actions](https://github.com/matlab-actions) build, test and deploy your toolbox as a part of [GitHub Action](https://docs.github.com/en/actions). MathWorks offers free licenses for public GitHub repositories, including support for Windows, Mac and Linux. If your GitHub repository is private, visit [this webpage](https://github.com/matlab-actions/setup-matlab?tab=readme-ov-file#use-matlab-batch-licensing-token). -Within a GitHub Action, you can invoke MATLAB's buildtool using [matlab-actions/run-command@v2](https://github.com/matlab-actions/run-command/). The build tasks that you already configured for local development like building MEX functions, running tests and packaging the toolbox can all be reused within your GitHub Workflow. +Within a GitHub Action, you can invoke MATLAB's buildtool using [matlab-actions/run-command@v2](https://github.com/matlab-actions/run-command/). The build tasks that you already configured for local development like building MEX functions, running tests and packaging the toolbox can all be reused within your GitHub Workflow. Our example uses a [matrix strategy](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/running-variations-of-jobs-in-a-workflow) in the GitHub Action to build MEX functions on different operating systems. ```yml -name: releaseWorkflow - -# This workflows creates a new (draft) release when a new git tag is pushed to GitHub. -on: - push: - tags: - - '*' - workflow_dispatch: - -jobs: - buildPackageAndRelease: - runs-on: windows-latest - - steps: - - name: Check out repository - uses: actions/checkout@v4 - - name: Set up MATLAB - uses: matlab-actions/setup-matlab@v2 - with: - release: R2024b - - name: Run script - uses: matlab-actions/run-command@v2 - with: - command: buildtool mex test package - - name: Create GitHub Release - uses: ncipollo/release-action@v1 - with: - draft: true - artifacts: "release/Arithmetic_Toolbox.mltbx" -``` - -### Multi platform MEX functions build using CI systems -You can use GitHub Actions' [matrix strategy](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/running-variations-of-jobs-in-a-workflow) to build MEX functions on different operating systems. We recommend splitting the workflow into two jobs, the first job creates the MEX functions for all the three platforms and publishes the MEX functions as a artifacts. The second job executes on a single operating system, creates a MLTBX file incorporating the platform dependent MEX functions and publishes the MLTBX to the release section on GitHub. - -```yml -name: releaseMultiplePlatform - -on: - push: - tags: - - '*' - workflow_dispatch: - -jobs: +# Exert from the YML file +Jobs: mexBuild: # Build MEX function on different operating systems using matrix strategy for operating systems. strategy: matrix: os: [ubuntu-latest, windows-latest, mac-latest] - MATLABVersion: [R2024b] runs-on: ${{ matrix.os }} + ... - steps: - - name: Check out repository + steps: + - name: Check out repository uses: actions/checkout@v4 - - name: Set up MATLAB + - name: Set up MATLAB uses: matlab-actions/setup-matlab@v2 + - name: Run build + uses: matlab-actions/run-build@v2 with: - release: ${{ matrix.MATLABVersion }} - - name: Run script - uses: matlab-actions/run-command@v2 - with: - command: buildtool mex - - name: Upload MEX - uses: actions/upload-artifact@v4 - with: - name: MEX_${{ matrix.os }} - path: toolbox/private - - packageAndRelease: - needs: mexBuild - runs-on: windows-latest - steps: - - name: Check out repository - uses: actions/checkout@v4 - - uses: actions/download-artifact@v4 - with: - name: MEX_ubuntu-latest - path: toolbox/private - - uses: actions/download-artifact@v4 - with: - name: MEX_windows-latest - path: toolbox/private - - name: Set up MATLAB - uses: matlab-actions/setup-matlab@v2 - with: - release: R2024b - - name: Run script - uses: matlab-actions/run-command@v2 - with: - command: buildtool test package - - name: Create GitHub Release - uses: ncipollo/release-action@v1 - with: - draft: true - artifacts: "release/Arithmetic_Toolbox.mltbx" + task: mex + ... ``` +For the full YML file refer to [`release.yml`](Add the link). + + - - ## Conclusion By following to these best practices, you'll create a reliable, maintainable, and user-friendly MATLAB toolbox that harnesses the full potential of MEX functions. Through effective project structure organization, build automation, and dependency management, you can focus on delivering great solutions to your users. From fdb342e7a4b9af67c579372e762d2e41e99f3942 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Thu, 19 Jun 2025 15:44:09 +0530 Subject: [PATCH 095/105] Added a mexTask that invokes the mex command directly --- MEX.md | 27 ++++++++++----------------- 1 file changed, 10 insertions(+), 17 deletions(-) diff --git a/MEX.md b/MEX.md index fb08c25..cbc3ffd 100644 --- a/MEX.md +++ b/MEX.md @@ -341,24 +341,17 @@ We welcome your input! For further details, suggestions, or to contribute, pleas ```matlab function plan = buildfile - % !! Works in 24a, introduced a dummy task called mex and made all the mex compile task as dependency. Not a great way to make it work. 24a does not have task collection. !! - plan = buildplan(); - - mexOutputFolder = fullfile("toolbox","private"); - - % Compile Cpp source code within cpp/*Mex into MEX functions - foldersToMex = plan.files(fullfile("cpp", "*Mex")).select(@isfolder); - mexTasks = string([]); - for folder = foldersToMex.paths - [~, folderName] = fileparts(folder); - plan("mex_"+folderName) = matlab.buildtool.tasks.MexTask(fullfile(folder, "**/*.cpp"), ... - mexOutputFolder, ... - Filename=folderName); - mexTasks(end+1) = "mex_" + folderName; + % Create a plan from the task functions + plan = buildplan(localfunctions); + plan("mex").Inputs = files(plan, fullfile("cpp","*Mex")); + plan("mex").Outputs = files(plan, fullfile("toolbox","private")); + +end + +function mexTask(context) + for f = context.Task.Inputs.paths + mex(fullfile(f, "*.cpp"), "-outdir", context.Task.Outputs.paths) end - plan("mex") = matlab.buildtool.Task; - plan("mex").Dependencies = mexTasks; - plan("mex").Description = "Build MEX functions"; end ``` From 7ca1b2438c9ee8affd9bf18ab19a379f446fdb23 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 23 Jun 2025 08:32:13 +0530 Subject: [PATCH 096/105] Added a MEX task for multiple single source MEX functions --- MEX.md | 24 +++++++++--------------- 1 file changed, 9 insertions(+), 15 deletions(-) diff --git a/MEX.md b/MEX.md index cbc3ffd..46191d0 100644 --- a/MEX.md +++ b/MEX.md @@ -358,23 +358,17 @@ end ```matlab function plan = buildfile -% !!Revise to work in 24a!! -% No task group in 24a, need to introduce a dummy mex task and make the actual tasks as a dependency - plan = buildplan(); - mexOutputFolder = fullfile("toolbox","private"); + % Create a plan from the task functions + plan = buildplan(localfunctions); + plan("mex").Inputs = files(plan, fullfile("cpp","mexfunctions","*.cpp")); + plan("mex").Outputs = files(plan, fullfile("toolbox","private")); + +end - % Compile all the folders inside cpp/*Mex.cpp into MEX functions - mexTasks = string([]); - filesToMex = plan.files(fullfile("cpp", "mexfunctions", "*.cpp")); - for cppFile = filesToMex.paths - [~, fileName] = fileparts(cppFile); - plan("mex_"+fileName) = matlab.buildtool.tasks.MexTask(cppFile, ... - mexOutputFolder); - mexTasks(end+1) = "mex_" + fileName; +function mexTask(context) + for mexSource = context.Task.Inputs.paths + mex(mexSource, "-outdir", context.Task.Outputs.paths) end - plan("mex") = matlab.buildtool.Task; - plan("mex").Dependencies = mexTasks; - plan("mex").Description = "Build MEX functions"; end ``` From 3fad78c7d79ed80576e18a3934339e07cd99843a Mon Sep 17 00:00:00 2001 From: Rob Purser Date: Fri, 27 Jun 2025 10:12:47 -0400 Subject: [PATCH 097/105] * Fixed Typos and minor formatting. * Removed commented out sections. Revised use of "YML" to "YAML" (which is [preferred](https://stackoverflow.com/questions/22268952/what-is-the-difference-between-yaml-and-yml-extension)) * Corrected copyright --- MEX.md | 88 +++++++++++++--------------------------------------------- 1 file changed, 20 insertions(+), 68 deletions(-) diff --git a/MEX.md b/MEX.md index cbc3ffd..37d38d9 100644 --- a/MEX.md +++ b/MEX.md @@ -9,11 +9,11 @@ Welcome to the MATLAB® MEX Best Practice guide, which extends [MATLAB Toolbo ## TL;DR - C++ code goes into the `cpp` folder. -- Each MEX functon is in its own folder with a `Mex` suffix -- Place the built MEX functions in a `private` folder inside your `toolbox` folder. MEX functions should only be called from within your toolbox in order to increase reliablity +- Each MEX function is in its own folder with a `Mex` suffix +- Place the built MEX functions in a `private` folder inside your `toolbox` folder. MEX functions should only be called from within your toolbox in order to increase reliability - External libraries are placed within a platform specific folder in the `private` folder and added to the system path -- We recommend using the [`mexhost`](https://www.mathworks.com/help/matlab/ref/mexhost.html) command to increase reliablity -- Use a [MexTask](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) in your [buildfile.m](https://www.mathworks.com/help/matlab/build-automation.html) for consistent builds +- We recommend using the [`mexhost`](https://www.mathworks.com/help/matlab/ref/mexhost.html) command to increase reliability +- Use a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html) in your [`buildfile.m`](https://www.mathworks.com/help/matlab/build-automation.html) for consistent builds ## Overview @@ -41,7 +41,7 @@ arithmetic/ Use the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command to compile `invertMex.cpp` into MEX functions. Place your compiled MEX functions in a `private` folder within the `toolbox` folder (see below). You need to provide the path of `invertMex.cpp` and path of the `private` folder as inputs to the `mex` command. -```matlab +``` matlab >> source = fullfile("cpp", "invertMex", "*.cpp"); >> destination = fullfile("toolbox", "private"); >> mex(source, "-outdir", destination, "-output", "invertMex") @@ -74,9 +74,9 @@ arithmetic/ Running the `mex` command for each MEX function can be tedious and error prone. MATLAB’s [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b, can automate this and many other repetitive processes for you. The following `buildfile.m` creates a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html), that builds your MEX functions. -```matlab +*This works in R2024b and later -- see the end of this document for a version supported in R2022a-R2024a.* +``` matlab function plan = buildfile - % Works in 24b. plan = buildplan(); @@ -94,9 +94,7 @@ function plan = buildfile end ``` - - -Our example toolbox adds a `buildfile.m` to automate the building of the MEX functions, on executing `buildtool mex`, the `invertMex.mexw64` gets created within the `private` folder. +Our example toolbox adds a `buildfile.m` to automate the building of the MEX functions. When you run "`buildtool mex`", the `invertMex.mexw64` gets created within the `private` folder. ``` text arithmetic/ @@ -116,11 +114,7 @@ arithmetic/ The above pattern extends to MEX functions that has multiple C++ source files. One of the source files must contain the [gateway](https://www.mathworks.com/help/matlab/matlab_external/gateway-routine.html) function. Place all the source files under a single folder. - - -Our example toolbox adds two .cpp files in the `subtractMex` folder, which builds to the `subtractMex.mexw64` binary in the `private` folder. `subtractNumber.m` provides a user accessible version that error checks before calling `SubtractMex`: +Our example toolbox adds two `.cpp` files in the `subtractMex` folder, which builds to the `subtractMex.mexw64` binary in the `private` folder. `subtractNumber.m` provides a user accessible version that error checks before calling `SubtractMex`: ``` text arithmetic/ @@ -164,7 +158,7 @@ arithmetic/ ``` The build file for building these MEX functions is slightly different. -```matlab +``` matlab function plan = buildfile % !!Revise to work in 24b!! plan = buildplan(); @@ -183,11 +177,6 @@ end ## Incorporating External Libraries - - - - - You can call libraries implemented in C++ using MEX functions. Since MEX source files are just C++ source code, they use standard C++ syntax to access external libraries. ### External Library Header Files (`.h`,`.hpp`) @@ -231,7 +220,7 @@ zlibStatic/ ### Calling a Dynamic Library [Dynamic libraries](https://www.learncpp.com/cpp-tutorial/a1-static-and-dynamic-libraries/) are required for running the MEX functions and must ship to the users. Place the binaries within the `private` folder under the `toolbox` folder to ensure the library gets shipped to the user. Use the [`-L`](https://www.mathworks.com/help/matlab/ref/mex.html#btw17rw-1-option1optionN) argument to the [`mex`](https://www.mathworks.com/help/matlab/ref/mex.html) command to specify the location and the name of the runtime library. -**Note:** If you have a choice between using a static or dynamic library with your MEX function, we recommend using a static library. Static libraries are incorprorated inside your MEX function, making your MEX function more robust and reliable. +**Note:** If you have a choice between using a static or dynamic library with your MEX function, we recommend using a static library. Static libraries are incorporated inside your MEX function, making your MEX function more robust and reliable. ``` text @@ -254,9 +243,8 @@ zlibShared/ ├───zlibShared.prj └───buildfile.m ``` - * For projects with complex dependencies, consider adopting dependency management tools like [Conan](https://conan.io/) which can significantly simplify library management across different platforms. -* Depending on the platform, dynamic libraries require adding their path to the operating system search path. These search paths are often set using environment variables. In case of C++ libraries you can use [`mexhost`](https://www.mathworks.com/help/matlab/ref/mexhost.html)'s `EnvironmentVariables` option to set-up the loader's path. The table below shows the environment variable for different operating systems. +* Depending on the platform, dynamic libraries require adding their path to the operating system search path. These search paths are often set using environment variables. In case of C++ libraries you can use the `EnvironmentVariables` option of [`mexhost`](https://www.mathworks.com/help/matlab/ref/mexhost.html) to set-up the loader's path. The table below shows the environment variable for different operating systems. | Operating system | Environment variable for loader's search path | | :-------- | :-------------------------------------------- | @@ -264,49 +252,13 @@ zlibShared/ | Windows | `PATH` | | Mac | `DYLD_LIBRARY_PATH` | - - - - - - - ## Automating Builds with GitHub Actions [MATLAB Actions](https://github.com/matlab-actions) build, test and deploy your toolbox as a part of [GitHub Action](https://docs.github.com/en/actions). MathWorks offers free licenses for public GitHub repositories, including support for Windows, Mac and Linux. If your GitHub repository is private, visit [this webpage](https://github.com/matlab-actions/setup-matlab?tab=readme-ov-file#use-matlab-batch-licensing-token). -Within a GitHub Action, you can invoke MATLAB's buildtool using [matlab-actions/run-command@v2](https://github.com/matlab-actions/run-command/). The build tasks that you already configured for local development like building MEX functions, running tests and packaging the toolbox can all be reused within your GitHub Workflow. Our example uses a [matrix strategy](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/running-variations-of-jobs-in-a-workflow) in the GitHub Action to build MEX functions on different operating systems. +Within a GitHub Action, you can invoke MATLAB's `buildtool` using [matlab-actions/run-command](https://github.com/matlab-actions/run-command/). The build tasks that you already configured for local development like building MEX functions, running tests and packaging the toolbox can all be reused within your GitHub Workflow. Our example uses a [matrix strategy](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/running-variations-of-jobs-in-a-workflow) in the GitHub Action to build MEX functions on different operating systems. -```yml -# Exert from the YML file +```yaml +# Excerpt from the YAML file Jobs: mexBuild: # Build MEX function on different operating systems using matrix strategy for operating systems. @@ -327,7 +279,7 @@ Jobs: task: mex ... ``` -For the full YML file refer to [`mexbuild.yml`](https://github.com/mathworks/arithmetic/blob/main/.github/workflows/mexbuild.yml). +For the full YAML file refer to [`mexbuild.yml`](https://github.com/mathworks/arithmetic/blob/main/.github/workflows/mexbuild.yml). @@ -337,9 +289,9 @@ By following to these best practices, you'll create a reliable, maintainable, an We welcome your input! For further details, suggestions, or to contribute, please [open an issue](https://github.com/mathworks/toolboxdesign/issues/new/choose). -## Appendix: Buildfiles for MATLAB R2024a +## Appendix: Buildfiles for MATLAB R2022a-R2024a -```matlab +``` matlab function plan = buildfile % Create a plan from the task functions plan = buildplan(localfunctions); @@ -356,7 +308,7 @@ end ``` -```matlab +``` matlab function plan = buildfile % !!Revise to work in 24a!! % No task group in 24a, need to introduce a dummy mex task and make the actual tasks as a dependency @@ -381,4 +333,4 @@ end --- [![CC-BY-4.0](images/cc-by-40.png)](https://creativecommons.org/licenses/by/4.0/) -Copyright © 2023-2025, The MathWorks, Inc. +Copyright © 2025, The MathWorks, Inc. From e48951e2920e5663a18ff4565693eec2252b62f8 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 1 Jul 2025 10:12:17 +0530 Subject: [PATCH 098/105] replaced run-command to run-build --- MEX.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index 46191d0..a625a7e 100644 --- a/MEX.md +++ b/MEX.md @@ -303,7 +303,7 @@ arithmetic/ ## Automating Builds with GitHub Actions [MATLAB Actions](https://github.com/matlab-actions) build, test and deploy your toolbox as a part of [GitHub Action](https://docs.github.com/en/actions). MathWorks offers free licenses for public GitHub repositories, including support for Windows, Mac and Linux. If your GitHub repository is private, visit [this webpage](https://github.com/matlab-actions/setup-matlab?tab=readme-ov-file#use-matlab-batch-licensing-token). -Within a GitHub Action, you can invoke MATLAB's buildtool using [matlab-actions/run-command@v2](https://github.com/matlab-actions/run-command/). The build tasks that you already configured for local development like building MEX functions, running tests and packaging the toolbox can all be reused within your GitHub Workflow. Our example uses a [matrix strategy](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/running-variations-of-jobs-in-a-workflow) in the GitHub Action to build MEX functions on different operating systems. +Within a GitHub Action, you can invoke MATLAB's buildtool using [matlab-actions/run-build](https://github.com/matlab-actions/run-build). The build tasks that you already configured for local development like building MEX functions, running tests and packaging the toolbox can all be reused within your GitHub Workflow. Our example uses a [matrix strategy](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/running-variations-of-jobs-in-a-workflow) in the GitHub Action to build MEX functions on different operating systems. ```yml # Exert from the YML file From ac054c02ab1c0c97d50e489bdb6b22c298d76b08 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 1 Jul 2025 11:24:56 +0530 Subject: [PATCH 099/105] Removed a comment for myself from the build script --- MEX.md | 1 - 1 file changed, 1 deletion(-) diff --git a/MEX.md b/MEX.md index fe23333..b6b63a1 100644 --- a/MEX.md +++ b/MEX.md @@ -160,7 +160,6 @@ The build file for building these MEX functions is slightly different. ``` matlab function plan = buildfile -% !!Revise to work in 24b!! plan = buildplan(); mexOutputFolder = fullfile("toolbox","private"); From 1919f77cc3247679fe6defae80e2f08b8c755d77 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 1 Jul 2025 11:26:47 +0530 Subject: [PATCH 100/105] merging the readme --- README.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/README.md b/README.md index c69b621..9c03df3 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,30 @@ We use the term “toolbox” here to mean a collection of reusable MATLAB code To make it easier to follow, we’ve created a fictitious toolbox for doing basic arithmetic: The Arithmetic Toolbox on [GitHub](https://github.com/mathworks/arithmetic). We’ll use this throughout to show how to apply these design principles. If you'd like to explore a complete toolbox that uses this structure, visit the [Climate Data Store Toolbox](https://github.com/mathworks/climatedatastore). +## TL;DR +* Root folder is a shortened version of the name of the toolbox +* Put all the materials that you plan to share with your users in `toolbox`sub-folder +* Provide a `GettingStarted.mlx` in `toolbox/doc` folder to help people get started +* Package and distribute your toolbox with MATLAB Toolbox files (`.mltbx`) +* Make your toolbox more robust by using tests, MATLAB Projects, source control, and `buildtool` +* Add the "Open in MATLAB Online" badge so users can try out your toolbox instantly + +Recommended file and folder structure: + +``` text +/ +| README.md # Overview of toolbox for users and developers +| license.txt # License file +└───images/ # Images for README.md +└───toolbox/ + | .m # Documented functions + | .m + ├───doc/ + | GettingStarted.mlx # Getting started guide + ├───examples/ # Examples (Live Scripts) + └───private/ # Implementation details, not intended for end users +``` + ## Topics * [Root Folder](#root-folder) From 1f6d929a58497598b371c168c0f5bf5dea7a97e5 Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 7 Jul 2025 14:56:38 +0530 Subject: [PATCH 101/105] changed task-> tasks in the workflow yml --- MEX.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index b6b63a1..8250743 100644 --- a/MEX.md +++ b/MEX.md @@ -275,7 +275,7 @@ Jobs: - name: Run build uses: matlab-actions/run-build@v2 with: - task: mex + tasks: mex ... ``` For the full YAML file refer to [`mexbuild.yml`](https://github.com/mathworks/arithmetic/blob/main/.github/workflows/mexbuild.yml). From b53ac8b5441f5c51bdd949b4930ac5fbc4251e3a Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Jul 2025 12:15:00 +0530 Subject: [PATCH 102/105] Added a link to advanced-ci-configuration-examples --- MEX.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index 8250743..62a7afa 100644 --- a/MEX.md +++ b/MEX.md @@ -252,7 +252,7 @@ zlibShared/ | Mac | `DYLD_LIBRARY_PATH` | ## Automating Builds with GitHub Actions -[MATLAB Actions](https://github.com/matlab-actions) build, test and deploy your toolbox as a part of [GitHub Action](https://docs.github.com/en/actions). MathWorks offers free licenses for public GitHub repositories, including support for Windows, Mac and Linux. If your GitHub repository is private, visit [this webpage](https://github.com/matlab-actions/setup-matlab?tab=readme-ov-file#use-matlab-batch-licensing-token). +[MATLAB Actions](https://github.com/matlab-actions) build, test and deploy your toolbox as a part of [GitHub Action](https://docs.github.com/en/actions). MathWorks offers free licenses for public GitHub repositories, including support for Windows, Mac and Linux. If your GitHub repository is private, visit [this webpage](https://github.com/matlab-actions/setup-matlab?tab=readme-ov-file#use-matlab-batch-licensing-token). You can also refer to [this GitHub repository](https://github.com/mathworks/advanced-ci-configuration-examples) for examples on advanced CI configurations from MathWorks. Within a GitHub Action, you can invoke MATLAB's buildtool using [matlab-actions/run-build](https://github.com/matlab-actions/run-build). The build tasks that you already configured for local development like building MEX functions, running tests and packaging the toolbox can all be reused within your GitHub Workflow. Our example uses a [matrix strategy](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/running-variations-of-jobs-in-a-workflow) in the GitHub Action to build MEX functions on different operating systems. From 3a48745454296b30d3de97c7f46102135b28183a Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Mon, 21 Jul 2025 13:55:24 +0530 Subject: [PATCH 103/105] Added a link to task group --- MEX.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/MEX.md b/MEX.md index 62a7afa..1b4c10f 100644 --- a/MEX.md +++ b/MEX.md @@ -72,7 +72,9 @@ arithmetic/ ### Automation using `buildtool` -Running the `mex` command for each MEX function can be tedious and error prone. MATLAB’s [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b, can automate this and many other repetitive processes for you. The following `buildfile.m` creates a [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html), that builds your MEX functions. +Running the `mex` command for each MEX function can be tedious and error prone. MATLAB’s [`buildtool`](https://www.mathworks.com/help/matlab/ref/buildtool.html), introduced in R2022b, can automate this and many other repetitive processes for you. The following `buildfile.m` uses [`MexTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.mextask-class.html), for building your MEX functions. + +We have defined the `mex` task as a [task groups](https://www.mathworks.com/help/matlab/matlab_prog/create-groups-of-similar-tasks.html), this allows us to build MEX functions out of multiple folders. *This works in R2024b and later -- see the end of this document for a version supported in R2022a-R2024a.* ``` matlab From 000df9c9d64d4837359981fb14fbe3a491ae25cf Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 22 Jul 2025 09:57:32 +0530 Subject: [PATCH 104/105] Updated the pre 24a buildfiles. added a clean task. --- MEX.md | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/MEX.md b/MEX.md index 1b4c10f..0bf82e3 100644 --- a/MEX.md +++ b/MEX.md @@ -294,16 +294,18 @@ We welcome your input! For further details, suggestions, or to contribute, pleas ``` matlab function plan = buildfile - % Create a plan from the task functions + % Create a plan using local functions plan = buildplan(localfunctions); - plan("mex").Inputs = files(plan, fullfile("cpp","*Mex")); - plan("mex").Outputs = files(plan, fullfile("toolbox","private")); - + plan("mex").Inputs.MexFolders = files(plan, fullfile("cpp", "*Mex")); + plan("mex").Inputs.Destination = fullfile("toolbox", "private"); + plan("mex").Outputs = fullfile("toolbox", "private", "*.mexw64"); + plan("clean") = matlab.buildtool.tasks.CleanTask; end function mexTask(context) - for f = context.Task.Inputs.paths - mex(fullfile(f, "*.cpp"), "-outdir", context.Task.Outputs.paths) + % Build multiple source MEX function within cpp/*Mex folders + for mexSourceFolder =context.Task.Inputs.MexFolders.paths + mex(fullfile(mexSourceFolder, "*.cpp"), "-outdir", context.Task.Inputs.Destination); end end ``` @@ -311,16 +313,18 @@ end ``` matlab function plan = buildfile - % Create a plan from the task functions + % Create a plan using local functions plan = buildplan(localfunctions); - plan("mex").Inputs = files(plan, fullfile("cpp","mexfunctions","*.cpp")); - plan("mex").Outputs = files(plan, fullfile("toolbox","private")); - + plan("mex").Inputs.MexFiles = files(plan, fullfile("cpp","mexfunctions/", "*.cpp")); + plan("mex").Inputs.Destination = fullfile("toolbox", "private"); + plan("mex").Outputs = fullfile("toolbox", "private", "*.mexw64"); + plan("clean") = matlab.buildtool.tasks.CleanTask; end function mexTask(context) - for mexSource = context.Task.Inputs.paths - mex(mexSource, "-outdir", context.Task.Outputs.paths) + % Build single source MEX functions within cpp/mexfunctions + for mexSourceFiles =context.Task.Inputs.MexFiles.paths + mex(mexSourceFiles, "-outdir", context.Task.Inputs.Destination); end end ``` From c92c72258bbde4df8daed3cc38abae95bade1f8c Mon Sep 17 00:00:00 2001 From: Bensingh Pancras Date: Tue, 22 Jul 2025 11:52:23 +0530 Subject: [PATCH 105/105] Added a clean task to the 24b build files. --- MEX.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/MEX.md b/MEX.md index 0bf82e3..0139c14 100644 --- a/MEX.md +++ b/MEX.md @@ -93,6 +93,7 @@ function plan = buildfile Filename=folderName); end plan("mex").Description = "Build MEX functions"; + plan("clean") = matlab.buildtool.tasks.CleanTask; end ``` @@ -173,6 +174,7 @@ function plan = buildfile mexOutputFolder); end plan("mex").Description = "Build MEX functions"; + plan("clean") = matlab.buildtool.tasks.CleanTask; end ```