Skip to content

intrinsic-test: Final code cleanup for the arm and common module #1884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions crates/intrinsic-test/src/arm/argument.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
use crate::arm::intrinsic::ArmIntrinsicType;
use crate::common::argument::Argument;

impl Argument<ArmIntrinsicType> {
pub fn type_and_name_from_c(arg: &str) -> (&str, &str) {
let split_index = arg
.rfind([' ', '*'])
.expect("Couldn't split type and argname");

(arg[..split_index + 1].trim_end(), &arg[split_index + 1..])
}
}
Comment on lines +1 to +12
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this really need its own file?

11 changes: 8 additions & 3 deletions crates/intrinsic-test/src/arm/json_parser.rs
Original file line number Diff line number Diff line change
Expand Up @@ -79,20 +79,25 @@ fn json_to_intrinsic(
) -> Result<Intrinsic<ArmIntrinsicType>, Box<dyn std::error::Error>> {
let name = intr.name.replace(['[', ']'], "");

let results = ArmIntrinsicType::from_c(&intr.return_type.value, target)?;
let mut results = ArmIntrinsicType::from_c(&intr.return_type.value)?;
results.set_metadata("target", target);

let args = intr
.arguments
.into_iter()
.enumerate()
.map(|(i, arg)| {
let arg_name = Argument::<ArmIntrinsicType>::type_and_name_from_c(&arg).1;
let (type_name, arg_name) = Argument::<ArmIntrinsicType>::type_and_name_from_c(&arg);
let metadata = intr.args_prep.as_mut();
let metadata = metadata.and_then(|a| a.remove(arg_name));
let arg_prep: Option<ArgPrep> = metadata.and_then(|a| a.try_into().ok());
let constraint: Option<Constraint> = arg_prep.and_then(|a| a.try_into().ok());
let mut ty = ArmIntrinsicType::from_c(type_name)
.unwrap_or_else(|_| panic!("Failed to parse argument '{arg}'"));
ty.set_metadata("target", target);

let mut arg = Argument::<ArmIntrinsicType>::from_c(i, &arg, target, constraint);
let mut arg =
Argument::<ArmIntrinsicType>::new(i, String::from(arg_name), ty, constraint);

// The JSON doesn't list immediates as const
let IntrinsicType {
Expand Down
48 changes: 28 additions & 20 deletions crates/intrinsic-test/src/arm/mod.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
mod argument;
mod compile;
mod config;
mod intrinsic;
mod json_parser;
mod types;

use std::fs::File;
use std::fs::{self, File};

use rayon::prelude::*;

Expand Down Expand Up @@ -69,9 +70,10 @@ impl SupportedArchitectureTest for ArmArchitectureTest {

let (chunk_size, chunk_count) = chunk_info(self.intrinsics.len());

let cpp_compiler = compile::build_cpp_compilation(&self.cli_options).unwrap();
let cpp_compiler_wrapped = compile::build_cpp_compilation(&self.cli_options);
Comment on lines -72 to +73
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we really should just panic if there is no compiler configured. What are we supposed to do in that scenario? We should not fail silently.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it could be

let Some(cpp_compiler) = compile::build_cpp_compilation(&self.cli_options) else { 
    panic!("no C compiler configured");
}

I guess.

Copy link
Contributor Author

@madhav-madhusoodanan madhav-madhusoodanan Jul 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a CLI option to only generate the C and Rust test files (without building or linking).
This is useful if I only need to get the test files without needing to setup the compilation or linking toolchains.

The relevant flag is --generate-only.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, ok, can you add a comment mentioning that flag?

(I think we should clean that up with a custom enum Mode { Compile(PathBuf), GenerateOnly } at some point)


let notice = &build_notices("// ");
fs::create_dir_all("c_programs").unwrap();
self.intrinsics
.par_chunks(chunk_size)
.enumerate()
Expand All @@ -81,9 +83,11 @@ impl SupportedArchitectureTest for ArmArchitectureTest {
write_mod_cpp(&mut file, notice, c_target, platform_headers, chunk).unwrap();

// compile this cpp file into a .o file
let output = cpp_compiler
.compile_object_file(&format!("mod_{i}.cpp"), &format!("mod_{i}.o"))?;
assert!(output.status.success(), "{output:?}");
if let Some(cpp_compiler) = cpp_compiler_wrapped.as_ref() {
let output = cpp_compiler
.compile_object_file(&format!("mod_{i}.cpp"), &format!("mod_{i}.o"))?;
assert!(output.status.success(), "{output:?}");
}

Ok(())
})
Expand All @@ -99,21 +103,25 @@ impl SupportedArchitectureTest for ArmArchitectureTest {
)
.unwrap();

// compile this cpp file into a .o file
info!("compiling main.cpp");
let output = cpp_compiler
.compile_object_file("main.cpp", "intrinsic-test-programs.o")
.unwrap();
assert!(output.status.success(), "{output:?}");

let object_files = (0..chunk_count)
.map(|i| format!("mod_{i}.o"))
.chain(["intrinsic-test-programs.o".to_owned()]);

let output = cpp_compiler
.link_executable(object_files, "intrinsic-test-programs")
.unwrap();
assert!(output.status.success(), "{output:?}");
// This is done because `cpp_compiler_wrapped` is None when
// the --generate-only flag is passed
if let Some(cpp_compiler) = cpp_compiler_wrapped.as_ref() {
// compile this cpp file into a .o file
info!("compiling main.cpp");
let output = cpp_compiler
.compile_object_file("main.cpp", "intrinsic-test-programs.o")
.unwrap();
assert!(output.status.success(), "{output:?}");

let object_files = (0..chunk_count)
.map(|i| format!("mod_{i}.o"))
.chain(["intrinsic-test-programs.o".to_owned()]);

let output = cpp_compiler
.link_executable(object_files, "intrinsic-test-programs")
.unwrap();
assert!(output.status.success(), "{output:?}");
}

true
}
Expand Down
16 changes: 10 additions & 6 deletions crates/intrinsic-test/src/arm/types.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
use std::collections::HashMap;

use super::intrinsic::ArmIntrinsicType;
use crate::common::cli::Language;
use crate::common::intrinsic_helpers::{IntrinsicType, IntrinsicTypeDefinition, Sign, TypeKind};
Expand Down Expand Up @@ -40,7 +42,7 @@ impl IntrinsicTypeDefinition for ArmIntrinsicType {
bit_len: Some(bl),
simd_len,
vec_len,
target,
metadata,
..
} = &self.0
{
Expand All @@ -50,7 +52,8 @@ impl IntrinsicTypeDefinition for ArmIntrinsicType {
""
};

let choose_workaround = language == Language::C && target.contains("v7");
let choose_workaround = language == Language::C
&& metadata.get("target").is_some_and(|val| val.contains("v7"));
format!(
"vld{len}{quad}_{type}{size}",
type = match k {
Expand Down Expand Up @@ -102,15 +105,16 @@ impl IntrinsicTypeDefinition for ArmIntrinsicType {
}
}

fn from_c(s: &str, target: &str) -> Result<Self, String> {
fn from_c(s: &str) -> Result<Self, String> {
const CONST_STR: &str = "const";
let metadata: HashMap<String, String> = HashMap::new();
if let Some(s) = s.strip_suffix('*') {
let (s, constant) = match s.trim().strip_suffix(CONST_STR) {
Some(stripped) => (stripped, true),
None => (s, false),
};
let s = s.trim_end();
let temp_return = ArmIntrinsicType::from_c(s, target);
let temp_return = ArmIntrinsicType::from_c(s);
temp_return.map(|mut op| {
op.ptr = true;
op.ptr_constant = constant;
Expand Down Expand Up @@ -151,7 +155,7 @@ impl IntrinsicTypeDefinition for ArmIntrinsicType {
bit_len: Some(bit_len),
simd_len,
vec_len,
target: target.to_string(),
metadata,
}))
} else {
let kind = start.parse::<TypeKind>()?;
Expand All @@ -167,7 +171,7 @@ impl IntrinsicTypeDefinition for ArmIntrinsicType {
bit_len,
simd_len: None,
vec_len: None,
target: target.to_string(),
metadata,
}))
}
}
Expand Down
36 changes: 9 additions & 27 deletions crates/intrinsic-test/src/common/argument.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,15 @@ impl<T> Argument<T>
where
T: IntrinsicTypeDefinition,
{
pub fn new(pos: usize, name: String, ty: T, constraint: Option<Constraint>) -> Self {
Argument {
pos,
name,
ty,
constraint,
}
}

pub fn to_c_type(&self) -> String {
self.ty.c_type()
}
Expand All @@ -36,14 +45,6 @@ where
self.constraint.is_some()
}

pub fn type_and_name_from_c(arg: &str) -> (&str, &str) {
let split_index = arg
.rfind([' ', '*'])
.expect("Couldn't split type and argname");

(arg[..split_index + 1].trim_end(), &arg[split_index + 1..])
}

/// The binding keyword (e.g. "const" or "let") for the array of possible test inputs.
fn rust_vals_array_binding(&self) -> impl std::fmt::Display {
if self.ty.is_rust_vals_array_const() {
Expand All @@ -62,25 +63,6 @@ where
}
}

pub fn from_c(
pos: usize,
arg: &str,
target: &str,
constraint: Option<Constraint>,
) -> Argument<T> {
let (ty, var_name) = Self::type_and_name_from_c(arg);

let ty =
T::from_c(ty, target).unwrap_or_else(|_| panic!("Failed to parse argument '{arg}'"));

Argument {
pos,
name: String::from(var_name),
ty: ty,
constraint,
}
}

fn as_call_param_c(&self) -> String {
self.ty.as_call_param_c(&self.name)
}
Expand Down
12 changes: 8 additions & 4 deletions crates/intrinsic-test/src/common/gen_rust.rs
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,12 @@ pub fn compile_rust_programs(toolchain: Option<&str>, target: &str, linker: Opti
/* If there has been a linker explicitly set from the command line then
* we want to set it via setting it in the RUSTFLAGS*/

// This is done because `toolchain` is None when
// the --generate-only flag is passed
if toolchain.is_none() {
return true;
}

trace!("Building cargo command");

let mut cargo_command = Command::new("cargo");
Expand All @@ -138,10 +144,8 @@ pub fn compile_rust_programs(toolchain: Option<&str>, target: &str, linker: Opti
// Do not use the target directory of the workspace please.
cargo_command.env("CARGO_TARGET_DIR", "target");

if let Some(toolchain) = toolchain
&& !toolchain.is_empty()
{
cargo_command.arg(toolchain);
if toolchain.is_some_and(|val| !val.is_empty()) {
cargo_command.arg(toolchain.unwrap());
}
cargo_command.args(["build", "--target", target, "--release"]);

Expand Down
9 changes: 7 additions & 2 deletions crates/intrinsic-test/src/common/intrinsic_helpers.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
use std::collections::HashMap;
use std::fmt;
use std::ops::Deref;
use std::str::FromStr;
Expand Down Expand Up @@ -121,7 +122,7 @@ pub struct IntrinsicType {
/// A value of `None` can be assumed to be 1 though.
pub vec_len: Option<u32>,

pub target: String,
pub metadata: HashMap<String, String>,
Comment on lines -124 to +125
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this a hashmap? Couldn't this be a struct instead? what values do you actually need to store in here for x86?

Copy link
Contributor Author

@madhav-madhusoodanan madhav-madhusoodanan Jul 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

X86 has a “TYPE” and “ETYPE” values for each argument of an intrinsic.

The “TYPE” tag is the type of the argument/return value as it appears in the C definition, while “ETYPE” gives information about how the argument will be used (eg: when using vector arguments, whether the operation would be U64 or FP16 based)

Both the information would be required, but it doesn’t make sense to create explicit struct members for it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, I didn't see what type this was in. Hmm, shouldn't the target-specific intrinsic type hold such information?

I'll say I am a bit surprised by the target being a part of IntrinsicType, that doesn't seem right.

}

impl IntrinsicType {
Expand Down Expand Up @@ -153,6 +154,10 @@ impl IntrinsicType {
self.ptr
}

pub fn set_metadata(&mut self, key: &str, value: &str) {
self.metadata.insert(key.to_string(), value.to_string());
}

pub fn c_scalar_type(&self) -> String {
match self.kind() {
TypeKind::Char(_) => String::from("char"),
Expand Down Expand Up @@ -322,7 +327,7 @@ pub trait IntrinsicTypeDefinition: Deref<Target = IntrinsicType> {
fn get_lane_function(&self) -> String;

/// can be implemented in an `impl` block
fn from_c(_s: &str, _target: &str) -> Result<Self, String>
fn from_c(_s: &str) -> Result<Self, String>
where
Self: Sized;

Expand Down