You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+16-4Lines changed: 16 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,14 +1,21 @@
1
1

2
2
3
+
# SML version 1.1
4
+
5
+
This is documentation for SML spec version `1.1`. For earlier versions browse the repository tags. Examples:
6
+
7
+
-[SML version 1.0](https://github.com/semanticdatalayer/SML/tree/v1.0)
8
+
3
9
# What is SML?
4
-
Semantic Modeling Language, or SML for short, encompasses over a decade of hands-on development, solving use cases for hundreds of customers across industries such as finance, healthcare, retail, manufacturing, CPG, and more. SML covers more than just tabular use cases. At its core, it is a multidimensional semantic modeling language that supports metrics, dimensions, hierarchies, semi-additive measures, many-to-many relationships, cell-based expressions, and much more.
10
+
11
+
Semantic Modeling Language, or SML for short, encompasses over a decade of hands-on development, solving use cases for hundreds of customers across industries such as finance, healthcare, retail, manufacturing, CPG, and more. SML covers more than just tabular use cases. At its core, it is a multidimensional semantic modeling language that supports metrics, dimensions, hierarchies, semi-additive measures, many-to-many relationships, cell-based expressions, and much more.
5
12
6
13
SML delivers on the following requirements:
7
14
8
15
1.**Object-oriented**: SML is an object-oriented language that promotes composability and inheritance. This allows semantic objects to be shared within other semantic objects and across organizations, supporting easy and consistent model-building.
9
16
2.**Comprehensive**: SML is based on more than a decade of modeling experience across various industry verticals and use cases. SML handles multi-dimensional constructs and serves as a superset of all other existing semantic modeling languages.
10
17
3.**Familiar**: SML is based on YAML, a widely adopted, human-readable, industry-standard syntax.
11
-
4.**CI/CD Friendly**: SML is code, so it is compatible with Git and CI/CD practices for version control, automated deployment, and software lifecycle management.
18
+
4.**CI/CD Friendly**: SML is code, so it is compatible with Git and CI/CD practices for version control, automated deployment, and software lifecycle management.
12
19
5.**Extensible**: SML syntax can be enhanced to support additional properties and features.
13
20
6.**Open**: SML is Apache open-sourced to support community innovation and is free to use in any application or use case.
14
21
@@ -25,6 +32,7 @@ We are or will be open-sourcing the following:
25
32
5.**Semantic Converters**: A CLI for translating other semantic modeling languages to and from SML, including Snowflake Cortex semantic models, Databricks UC Metrics, and Power BI semantic models.
26
33
27
34
## SML Example
35
+
28
36
The following is an example of an SML `model` object:
29
37
30
38
```
@@ -59,6 +67,7 @@ metrics:
59
67
```
60
68
61
69
## SML Object Hierarchy
70
+
62
71
The following graphic illustrates the key SML objects and their relationships:
63
72
64
73
```mermaid
@@ -85,7 +94,7 @@ The following sections describe the different SML object types as well
85
94
as the properties available for each:
86
95
87
96
-[Catalog](sml-reference/catalog.md) - Defines the control file for a SML repository. It contains all repository-level definitions.
88
-
-[Package](sml-reference/package.md) - Defines additional Git repositories references whose objects can be used in the current repository.
97
+
-[Package](sml-reference/package.md) - Defines additional Git repositories references whose objects can be used in the current repository.
89
98
-[Model](sml-reference/model.md) - Defines the logical, business-friendly representation on top of the physical data.
90
99
-[Dimension](sml-reference/dimension.md) - Defines the logical collection of attributes and hierarchies for supporting drill-down.
91
100
-[Row Security](sml-reference/row-security.md) - Defines row-level data access rules for users and groups.
@@ -102,15 +111,18 @@ as the properties available for each:
102
111
## Model Library
103
112
104
113
### Tutorial Models
114
+
105
115
1.[Internet Sales Model](https://github.com/semanticdatalayer/sml-models-tutorials-internet-sales) - a simple, single-fact model derived from the fictitious AdventureWorks retail dataset.
106
116
2.[World Wide Importers Model](https://github.com/semanticdatalayer/sml-models-tutorials-ww-importers) - a more complex, multi-fact model representing a fictional wholesale and distribution company.
107
117
3.[TPC-DS Model](https://github.com/semanticdatalayer/sml-models-tutorials-tpcds) - a complex, multi-fact model that encodes the [TPC-DS](https://www.tpc.org/tpcds/) benchmark model in SML.
108
118
4.[TPC-H Model](https://github.com/semanticdatalayer/sml-models-tutorials-tpch) - a complex, multi-fact model that encodes the [TPC-H](https://www.tpc.org/tpch/) benchmark model in SML.
109
-
5.[AdventureWorks2012 Model](https://github.com/semanticdatalayer/sml-models-tutorials-adventureworks2012) - the standard Microsoft SSAS tutorial in SML.
119
+
5.[AdventureWorks2012 Model](https://github.com/semanticdatalayer/sml-models-tutorials-adventureworks2012) - the standard Microsoft SSAS tutorial in SML.
110
120
111
121
### Data Warehouse Usage/Cost Models
122
+
112
123
1.[Snowflake Usage Model](https://github.com/semanticdatalayer/sml-models-usage-snowflake) - a semantic model for analyzing Snowflake credit and data warehouse usage.
113
124
114
125
### Marketplace Models
126
+
115
127
1.[Snowplow Digital Analytics Model](https://github.com/semanticdatalayer/sml-models-snowplow) - Snowplow empowers organizations to create a scalable, first-party data foundation so marketing and data teams can effectively analyze and tackle Customer 360 use cases.
116
128
2.[CRISP CPG Retail and Distributor Data Model](https://github.com/semanticdatalayer/sml-models-crisp-cpg-retail) - Crisp connects to over 40 leading U.S. retailers and distributors.
Copy file name to clipboardExpand all lines: sml-reference/dimension.md
+47Lines changed: 47 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -269,6 +269,7 @@ namespace Dimensions{
269
269
Boolean is_aggregatable
270
270
Boolean exclude_from_fact_agg
271
271
String time_unit
272
+
Int constraint_translation_rank
272
273
Array~String~ allowed_calcs_for_dma
273
274
CustomEmptyMember custom_empty_member
274
275
String folder
@@ -308,12 +309,15 @@ namespace Dimensions{
308
309
String label
309
310
String description
310
311
String folder
312
+
Number precedence
311
313
Array~CalculatedMembers~ calculated_members
314
+
Boolean is_hidden
312
315
}
313
316
class CalculatedMembers{
314
317
String unique_name
315
318
String description
316
319
String format
320
+
Boolean is_hidden
317
321
String expression
318
322
Boolean use_input_metric_format
319
323
String template
@@ -541,13 +545,36 @@ Defines the individual calculated members in the group.
541
545
542
546
A description of the calculation group.
543
547
548
+
## is_hidden
549
+
550
+
-**Type:** boolean
551
+
-**Required:** N
552
+
553
+
Determines whether the calculation group is visible in BI tools.
554
+
555
+
Supported values:
556
+
557
+
-`false` (default)
558
+
-`true`
559
+
544
560
## folder
545
561
546
562
-**Type:** string
547
563
-**Required:** N
548
564
549
565
The name of the folder in which the calculation group is displayed in BI tools.
550
566
567
+
## precedence
568
+
569
+
-**Type:** number
570
+
-**Required:** N
571
+
572
+
Update to "Precedence" explicitly defines the order of Calculation Group evaluation, making it consistent across BI tools.
573
+
574
+
Supported values:
575
+
576
+
- Integer and floating point numbers
577
+
551
578
# Calculated Members Properties
552
579
553
580
## unique_name
@@ -577,6 +604,18 @@ Supported templates:
577
604
578
605
If you do not want to use a built-in template, you can define a custom expression using the `expression` property (see below).
579
606
607
+
## is_hidden
608
+
609
+
-**Type:** boolean
610
+
-**Required:** N
611
+
612
+
Determines whether the attribute is visible in BI tools.
613
+
614
+
Supported values:
615
+
616
+
-`false` (default)
617
+
-`true`
618
+
580
619
## expression
581
620
582
621
-**Type:** string
@@ -1188,6 +1227,14 @@ If the key consists of one column, the values in that column must be
1188
1227
unique. If the key is a compound key, the columns together must provide
1189
1228
unique values.
1190
1229
1230
+
## constraint_translation_rank
1231
+
1232
+
-**Type:** integer
1233
+
-**Required:** N
1234
+
-**Range:** should be a valid 32 bit integer
1235
+
1236
+
Defines the translation of dimension filter constraints into fact table partition column constraints. This can significantly improve query performance for cases where fact-based aggregates are not used.
For example, if you wanted to use the prefix **Order**, you would set
396
396
`role_play` to `"Order {0}"`.
397
397
398
+
### constraint_translation
399
+
400
+
-**Type:** object
401
+
-**Required:** N
402
+
403
+
Defines the translation of dimension filter constraints into fact table partition column constraints. This can significantly improve query performance for cases where fact-based aggregates are not used.
404
+
405
+
Supported properties:
406
+
407
+
-`level`: String, required. Indicates the dimension level to which the constraint translation applies.
408
+
-`from_columns`: Array, required. Lists the column(s) in the dataset that should be used for the join.
409
+
410
+
If the `constraint_translation` property is defined, a corresponding `constraint_translation_rank` must be present in the associated level.
411
+
398
412
## metrics
399
413
400
414
-**Type:** array
@@ -440,7 +454,7 @@ analysts with the entire data model, you can make specific dimensions,
440
454
hierarchies, levels, secondary attributes, measures, and calculated
441
455
measures invisible to them.
442
456
443
-
**Note:** We recommend that you add perspectives *after* a model has
457
+
**Note:** We recommend that you add perspectives _after_ a model has
444
458
been fully tested. Although you can edit a model after adding
445
459
perspectives, any changes might require you to update the perspectives
446
460
to hide new objects that would otherwise be visible to all users.
@@ -476,9 +490,9 @@ perspective.
476
490
A list of the specific dimensions and their hierarchies to be hidden in the
477
491
perspective.
478
492
479
-
By default, all objects within a dimension are visible. The lowest granularity objects specified are
480
-
hidden and the objects above it are not. Hiding a level in a hierarchy hides all levels below it.
481
-
Hiding a hierarchy hides all levels in it. Hiding a dimension hides all objects within it including hierarchies
493
+
By default, all objects within a dimension are visible. The lowest granularity objects specified are
494
+
hidden and the objects above it are not. Hiding a level in a hierarchy hides all levels below it.
495
+
Hiding a hierarchy hides all levels in it. Hiding a dimension hides all objects within it including hierarchies
482
496
and secondary attributes. If a dimension is not hidden, secondary attributes can be hidden individually.
483
497
484
498
Supported properties:
@@ -488,8 +502,10 @@ Supported properties:
488
502
489
503
-`hierarchies`: Array, optional. A list of the specific hierarchies
490
504
within the dimension to hide in the perspective. Supported properties:
491
-
-`name`: String, required. The name of the hierarchy.
492
-
-`levels`: Array, optional. Defines a single level in the hierarchy to be hidden in the perspective. All levels below the specified level will also be hidden. Only one level should be provided.
505
+
506
+
-`name`: String, required. The name of the hierarchy.
507
+
-`level`: String, optional. Defines a single level in the hierarchy to be hidden in the perspective. All levels below the specified level will also be hidden.
508
+
-`levels`: Array, optional. ⚠️ **DEPRECATED** use `level` instead.
493
509
494
510
-`secondary_attributes`: Array, optional. A list of the dimension's
495
511
secondary attributes to hide in the perspective.
@@ -643,22 +659,22 @@ Supported properties:
643
659
determines whether it should be defined on the key column, name
644
660
column, or both. Supported values: `name`, `key`, `name+key`
645
661
646
-
When the engine builds an instance of this aggregate, it creates
647
-
a partition for each combination of values in the dimensional
648
-
attributes. The number of partitions depends on the
649
-
left-to-right order of the attributes, as well as the number of
650
-
values for each attribute.
662
+
When the engine builds an instance of this aggregate, it creates
663
+
a partition for each combination of values in the dimensional
664
+
attributes. The number of partitions depends on the
665
+
left-to-right order of the attributes, as well as the number of
666
+
values for each attribute.
651
667
652
-
Essentially, the partitioning key functions as a `GROUP BY`
653
-
column. Queries against the aggregate must use this dimensional
654
-
attribute in a `WHERE` clause. A good candidate for a
655
-
partitioning key is a set of dimensional attributes that
656
-
together have a few hundred to under 1000 value combinations.
668
+
Essentially, the partitioning key functions as a `GROUP BY`
669
+
column. Queries against the aggregate must use this dimensional
670
+
attribute in a `WHERE` clause. A good candidate for a
671
+
partitioning key is a set of dimensional attributes that
672
+
together have a few hundred to under 1000 value combinations.
657
673
658
674
-`distribution`: String, optional. The distribution keys to use when
659
675
creating the aggregate table. If your aggregate data warehouse
660
676
supports distribution keys, then the semantic engine uses the specified keys when
661
-
creating the aggregate table.
677
+
creating the aggregate table.
662
678
663
679
## partitions
664
680
@@ -733,7 +749,7 @@ Supported properties:
733
749
-`allow_peer_aggs`: Boolean, optional. Enables aggregation on data
734
750
derived from datasets in data warehouses that are different from the
735
751
source dataset.
736
-
-`allow_preferred_aggs`: Boolean, optional. Allow aggregates to be built
752
+
-`allow_preferred_aggs`: Boolean, optional. Allow aggregates to be built
737
753
in preferred storage.
738
754
-`create_hinted_aggregate`: Boolean, options. Enables the creation of
0 commit comments