Support for Slurm 24.11.x (#361)

* slurm_get_errno was removed, just use errno directly

* "List" type was removed, use list_t now

* qos: previous bool members were moved into a single flags variable

* bump version to 24.11.0

* JOB_UPDATE_DB was removed

* add new headers for 24.11

* make pyslurm.pyx compatible with 24.11.x

- print_job_info_msg removed - the underlying API call was removed by
  the slurm devs
- change List to list_t
- slurm_get_errno was removed, use errno directly
- set cpus_allocated to 0 and cpus_alloc_layout to empty list when
  retrieving jobs - API calls were removed (and this class is deprecated
  anyway)

* remove slurm_seterrno calls

this API was removed from slurm

* Update pyslurm.spec versions

* Update CHANGELOG

Author: tazend

CHANGELOG.md

@@ -5,7 +5,7 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## Unreleased on the [24.5.x](https://github.com/PySlurm/pyslurm/tree/24.5.x) branch
+## Unreleased on the [24.11.x](https://github.com/PySlurm/pyslurm/tree/24.11.x) branch
 
 ### Added
 
@@ -16,84 +16,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
     - `pyslurm.db.QualityOfService`
     - `pyslurm.db.QualitiesOfService`
 
-## [24.5.1](https://github.com/PySlurm/pyslurm/releases/tag/v24.5.1) - 2024-12-27
+## [24.11.0](https://github.com/PySlurm/pyslurm/releases/tag/v24.11.0) - 2024-12-30
 
 ### Added
 
-- Added `stats` attribute to both `pyslurm.Job`, `pyslurm.Jobs` and
-  `pyslurm.db.Jobs`
-- Added `pids` attribute to `pyslurm.Job` which contains Process-IDs of the Job
-  organized by node-name
-- Added `load_stats` method to `pyslurm.Job` and `pyslurm.Jobs` classes.
-  Together with the `stats` and `pids` attributes mentioned above, it is now
-  possible to fetch live statistics (like sstat)
-- Switch to link with `libslurmfull.so` instead of `libslurm.so`<br>
-  This change really has no impact from a user perspective. Everything will
-  keep working the same, except that Slurms more internal library
-  `libslurmfull.so` is linked with (which is located alongside the plugins
-  inside the `slurm` directory, which itself is next to `libslurm.so`)<br>
-  Why the change? Because it will likely make development easier. It allows
-  access to more functions that might be needed in some places, without
-  completely having to implement them on our own. Implementing the
-  live-statistics feature, so basically `sstat`, is for example not possible
-  with `libslurm.so` <br>
-  You can keep providing the directory where `libslurm.so` resided as
-  `$SLURM_LIB_DIR` to pyslurm, and it will automatically find `libslurmfull.so`
-  from there.
-- Added `run_time_remaining` and `elapsed_cpu_time` attributes to `pyslurm.JobStep`
-- Added `run_time_remaining` attribute to `pyslurm.Job`
-
-### Fixed
-
-- Fixed `total_cpu_time`, `system_cpu_time` and `user_cpu_time` not getting
-  calculated correctly for Job statistics
-- Actually make sure that `avg_cpu_time`, `min_cpu_time`, `total_cpu_time`,
-  `system_cpu_time` and `user_cpu_time` are integers, not float.
-
-### Changed
-
-- Breaking: rename `cpu_time` to `elapsed_cpu_time` in `pyslurm.Job` and
-  `pyslurm.Jobs` classes
-- Breaking: rename attribute `alloc_cpus` to just `cpus` in `pyslurm.JobStep`
-- Breaking: removed the following attributes from `pyslurm.db.Jobs`:<br>
-    * `consumed_energy`
-    * `disk_read`
-    * `disk_write`
-    * `page_faults`
-    * `resident_memory`
-    * `virtual_memory`
-    * `elapsed_cpu_time`
-    * `total_cpu_time`
-    * `user_cpu_time`
-    * `system_cpu_time`
-- The removed attributes above are now all available within the `stats`
-  attribute, which is of type `pyslurm.db.JobStatistics`
-- Renamed the original class of `pyslurm.db.JobStatistics` to
-  `pyslurm.db.JobStepStatistics`.<br>
-  All this class contains is really mostly applicable only to Steps, but
-  doesn't fully apply at the Job Level.<br>
-  Therefore, the new `pyslurm.db.JobStatistics` class only contains all
-  statistics that make sense at the Job-level.
-- return `1` as a value for the `cpus` attribute in `pyslurm.db.Job` when there
-  is no value set from Slurm's side.
-
-### Removed
-
-- Removed `pyslurm.version()` function. Should use `__version__` attribute directly.
-- Removed `--slurm-lib` and `--slurm-inc` parameters to `setup.py`.<br>
-  `SLURM_LIB_DIR` and `SLURM_INCLUDE_DIR` environment variables should be used instead.
-
-## [24.5.0](https://github.com/PySlurm/pyslurm/releases/tag/v24.5.0) - 2024-11-16
-
-### Added
-
-- Support for Slurm 24.5.x
-- add `power_down_on_idle` attribute to `pyslurm.Partition` class
-
-### Changed
-
-- bump minimum Cython version to 0.29.37
-
-### Removed
-
-- Removed `power_options` from `JobSubmitDescription`
+- Support for Slurm 24.11.x

README.md

@@ -8,16 +8,16 @@ pyslurm is the Python client library for the [Slurm Workload Manager](https://sl
 * [Python](https://www.python.org) - >= 3.6
 * [Cython](https://cython.org) - >= 0.29.37
 
-This Version is for Slurm 24.05.x
+This Version is for Slurm 24.11.x
 
 ## Versioning
 
 In pyslurm, the versioning scheme follows the official Slurm versioning. The
 first two numbers (`MAJOR.MINOR`) always correspond to Slurms Major-Release,
-for example `24.05`.
+for example `24.11`.
 The last number (`MICRO`) is however not tied in any way to Slurms `MICRO`
 version, but is instead PySlurm's internal Patch-Level. For example, any
-pyslurm 24.05.X version should work with any Slurm 24.05.X release.
+pyslurm 24.11.X version should work with any Slurm 24.11.X release.
 
 ## Installation
 
@@ -29,8 +29,8 @@ the corresponding paths to the necessary files.
 You can specify those with environment variables (recommended), for example:
 
 ```shell
-export SLURM_INCLUDE_DIR=/opt/slurm/24.05/include
-export SLURM_LIB_DIR=/opt/slurm/24.05/lib
+export SLURM_INCLUDE_DIR=/opt/slurm/24.11/include
+export SLURM_LIB_DIR=/opt/slurm/24.11/lib
 ```
 
 Then you can proceed to install pyslurm, for example by cloning the Repository:

pyslurm.spec

@@ -1,7 +1,7 @@
 %define python3_pkgversion 3.11
 
 Name:            python-pyslurm
-Version:         24.5.1
+Version:         24.11.0
 %define rel      1
 Release:         %{rel}%{?dist}
 Summary:         Python interface to Slurm
@@ -15,8 +15,8 @@ BuildRequires:   python%{python3_pkgversion}-wheel
 BuildRequires:   python%{python3_pkgversion}-Cython
 BuildRequires:   python%{python3_pkgversion}-packaging
 BuildRequires:   python-rpm-macros
-BuildRequires:   slurm-devel >= 24.05.0
-BuildRequires:   slurm >= 24.05.0
+BuildRequires:   slurm-devel >= 24.11.0
+BuildRequires:   slurm >= 24.11.0
 Requires:        python%{python3_pkgversion}
 
 %description

pyslurm/core/error.pyx

@@ -24,7 +24,7 @@
 
 from pyslurm.utils cimport cstr
 from pyslurm cimport slurm
-from pyslurm.slurm cimport slurm_get_errno
+cimport libc.errno
 
 
 def slurm_strerror(errno):
@@ -36,7 +36,7 @@ def slurm_strerror(errno):
             returned.
 
     Returns:
-        (str): String representation of errno.  
+        (str): String representation of errno.
     """
     return cstr.to_unicode(slurm.slurm_strerror(errno))
 
@@ -47,7 +47,7 @@ def slurm_errno():
     Returns:
         (int): Current slurm errno
     """
-    return slurm_get_errno()
+    return libc.errno.errno
 
 
 def get_last_slurm_error():
@@ -75,7 +75,7 @@ class RPCError(PyslurmError):
     Args:
         errno (int):
             A slurm error number returned by RPC functions. Default is None,
-            which will get the last slurm error automatically. 
+            which will get the last slurm error automatically.
         msg (str):
             An optional, custom error description. If this is set, the errno
             will not be translated to its string representation.
@@ -89,7 +89,7 @@ class RPCError(PyslurmError):
     """
     def __init__(self, errno=slurm.SLURM_ERROR, msg=None):
         self.msg = msg
-        self.errno = errno 
+        self.errno = errno
 
         if not msg:
             if errno == slurm.SLURM_ERROR:

pyslurm/core/partition.pxd

@@ -27,6 +27,7 @@ from libc.string cimport memcpy, memset
 from pyslurm cimport slurm
 from libc.stdint cimport uint8_t, uint16_t, uint32_t, uint64_t
 from pyslurm.slurm cimport (
+    list_t,
     partition_info_msg_t,
     job_defaults_t,
     delete_part_msg_t,

pyslurm/core/partition.pyx

@@ -807,7 +807,7 @@ def _preempt_mode_int_to_str(mode, slurmctld.Config slurm_conf):
         return cstr.to_unicode(slurm_preempt_mode_string(mode))
 
 
-cdef _extract_job_default_item(typ, slurm.List job_defaults_list):
+cdef _extract_job_default_item(typ, list_t *job_defaults_list):
     cdef:
         job_defaults_t *default_item
         SlurmList job_def_list

pyslurm/db/qos.pxd

@@ -30,7 +30,7 @@ from pyslurm.slurm cimport (
     slurmdb_destroy_qos_cond,
     slurmdb_qos_get,
     slurm_preempt_mode_num,
-    List,
+    list_t,
     try_xmalloc,
 )
 from pyslurm.db.util cimport (
@@ -40,8 +40,9 @@ from pyslurm.db.util cimport (
 )
 from pyslurm.db.connection cimport Connection
 from pyslurm.utils cimport cstr
+from pyslurm.utils.uint cimport u16_set_bool_flag
 
-cdef _set_qos_list(List *in_list, vals, QualitiesOfService data)
+cdef _set_qos_list(list_t **in_list, vals, QualitiesOfService data)
 
 
 cdef class QualitiesOfService(dict):

pyslurm/db/qos.pyx

@@ -124,7 +124,8 @@ cdef class QualityOfServiceFilter:
         make_char_list(&ptr.id_list, self.ids)
         make_char_list(&ptr.description_list, self.descriptions)
         ptr.preempt_mode = self._parse_preempt_modes()
-        ptr.with_deleted = 1 if bool(self.with_deleted) else 0
+        u16_set_bool_flag(&ptr.flags, self.with_deleted,
+                          slurm.QOS_COND_FLAG_WITH_DELETED)
 
 
 cdef class QualityOfService:
@@ -223,6 +224,6 @@ def _validate_qos_single(qid, QualitiesOfService data):
     raise ValueError(f"Invalid QOS specified: {qid}")
 
 
-cdef _set_qos_list(List *in_list, vals, QualitiesOfService data):
+cdef _set_qos_list(list_t **in_list, vals, QualitiesOfService data):
     qos_ids = _qos_names_to_ids(vals, data)
     make_char_list(in_list, qos_ids)

pyslurm/db/util.pxd

@@ -24,7 +24,7 @@ from pyslurm cimport slurm
 from pyslurm.utils cimport cstr
 from pyslurm.slurm cimport (
     list_itr_t,
-    List,
+    list_t,
     slurm_list_iterator_create,
     slurm_list_iterator_destroy,
     slurm_list_iterator_reset,
@@ -37,9 +37,9 @@ from pyslurm.slurm cimport (
     slurm_xfree_ptr,
 )
 
-cdef slurm_list_to_pylist(List in_list)
-cdef make_char_list(List *in_list, vals)
-cdef qos_list_to_pylist(List in_list, qos_data)
+cdef slurm_list_to_pylist(list_t *in_list)
+cdef make_char_list(list_t **in_list, vals)
+cdef qos_list_to_pylist(list_t *in_list, qos_data)
 
 
 cdef class SlurmListItem:
@@ -51,16 +51,16 @@ cdef class SlurmListItem:
 
 cdef class SlurmList:
     cdef:
-        List info
+        list_t *info
         list_itr_t *itr
 
     cdef readonly:
         owned
         int itr_cnt
         int cnt
-    
+
     @staticmethod
-    cdef SlurmList wrap(List, owned=*)
+    cdef SlurmList wrap(list_t *li, owned=*)
 
     @staticmethod
     cdef SlurmList create(slurm.ListDelF delf, owned=*)

pyslurm/db/util.pyx

@@ -23,7 +23,7 @@
 # cython: language_level=3
 
 
-cdef make_char_list(List *in_list, vals):
+cdef make_char_list(list_t **in_list, vals):
     if not vals:
         return None
 
@@ -39,11 +39,11 @@ cdef make_char_list(List *in_list, vals):
     slist.owned = False
 
 
-cdef slurm_list_to_pylist(List in_list):
+cdef slurm_list_to_pylist(list_t *in_list):
     return SlurmList.wrap(in_list, owned=False).to_pylist()
 
 
-cdef qos_list_to_pylist(List in_list, qos_data):
+cdef qos_list_to_pylist(list_t *in_list, qos_data):
     if not in_list:
         return []
 
@@ -53,7 +53,7 @@ cdef qos_list_to_pylist(List in_list, qos_data):
 
 
 cdef class SlurmListItem:
-    
+
     def __cinit__(self):
         self.data = NULL
 
@@ -138,7 +138,7 @@ cdef class SlurmList:
         return wrapper
 
     @staticmethod
-    cdef SlurmList wrap(List li, owned=True):
+    cdef SlurmList wrap(list_t *li, owned=True):
         cdef SlurmList wrapper = SlurmList.__new__(SlurmList)
         if not li:
             return wrapper