Slurm Workload Manager -

    .SH "DESCRIPTION"
    \fBsacctmgr\fR is used to view or modify Slurm account information.
    The account information is maintained within a database with the interface
    being provided by \fBslurmdbd\fR (Slurm Database daemon).
    This database can serve as a central storehouse of user and
    computer information for multiple computers at a single site.
    Slurm account information is recorded based upon four parameters
    that form what is referred to as an \fIassociation\fR.
    These parameters are \fIuser\fR, \fIcluster\fR, \fIpartition\fR, and
    \fIaccount\fR. \fIuser\fR is the login name.
    \fIcluster\fR is the name of a Slurm managed cluster as specified by
    the \fIClusterName\fR parameter in the \fIslurm.conf\fR configuration file.
    \fIpartition\fR is the name of a Slurm partition on that cluster.
    \fIaccount\fR is the account for a job.
    The intended mode of operation is to initiate the \fBsacctmgr\fR command,
    add, delete, modify, and/or list \fIassociation\fR records then
    commit the changes and exit.

    \fBNOTE\fR: The contents of Slurm's database are maintained in lower case.
    This may result in some \f3sacctmgr\fP output differing from that of other
    Slurm commands.

    .SH "OPTIONS"

    .TP
    \fB\-s\fR, \fB\-\-associations\fR
    Use with show or list to display associations with the entity.
    This is equivalent to the \fBassociations\fR command.
    .IP

    .TP
    \fB\-h\fR, \fB\-\-help\fR
    Print a help message describing the usage of \fBsacctmgr\fR.
    This is equivalent to the \fBhelp\fR command.
    .IP

    .TP
    \fB\-i\fR, \fB\-\-immediate\fR
    Commit changes immediately without asking for confirmation.
    .IP

    .TP
    \f3\-\-json\fP, \f3\-\-json\fP=\fIlist\fR, \f3\-\-json\fP=<\fIdata_parser\fR>
    Dump information as JSON using the default data_parser plugin or explicit
    data_parser with parameters. Sorting and formatting arguments will be ignored.
    This option is not available for every command.
    .IP

    .TP
    \fB\-n\fR, \fB\-\-noheader\fR
    No header will be added to the beginning of the output.
    Print no messages other than error messages.
    This is equivalent to the \fBquiet\fR command.
    .IP

    .TP
    \fB\-r\fR, \fB\-\-readonly\fR
    Makes it so the running sacctmgr cannot modify accounting information.
    The \fBreadonly\fR option is for use within interactive mode.
    .IP

    .TP
    \f3\-\-yaml\fP, \f3\-\-yaml\fP=\fIlist\fR, \f3\-\-yaml\fP=<\fIdata_parser\fR>
    Dump information as YAML using the default data_parser plugin or explicit
    data_parser with parameters. Sorting and formatting arguments will be ignored.
    This option is not available for every command.
    .IP

    .TP
    \fB\-v\fR, \fB\-\-verbose\fR
    Enable detailed logging.
    This is equivalent to the \fBverbose\fR command.
    .IP

    .TP
    \fB\-V\fR , \fB\-\-version\fR
    Display version number.
    This is equivalent to the \fBversion\fR command.
    .IP

    .SH "COMMANDS"

    .TP
    \fBadd\fR <\fIENTITY\fR> <\fISPECS\fR>
    Add an entity.
    Identical to the \fBcreate\fR command.
    .IP

    .TP
    \fBarchive\fR {dump|load} <\fISPECS\fR>
    Write database information to a flat file or load information that has
    previously been written to a file.
    .IP

    .TP
    \fBclear stats\fR
    Clear the server statistics.
    .IP

    .TP
    \fBcreate\fR <\fIENTITY\fR> <\fISPECS\fR>
    Add an entity.
    Identical to the \fBadd\fR command.
    .TP
    \fBhelp\fP
    Display a description of sacctmgr options and commands.
    .IP

    .TP
    \fBlist\fR <\fIENTITY\fR> [<\fISPECS\fR>]
    Display information about the specified entity.
    By default, all entries are displayed, you can narrow results by
    specifying SPECS in your query.
    Identical to the \fBshow\fR command.
    .IP

    .TP
    \fBload\fR <\fIFILENAME\fR>
    Load cluster data from the specified file. This is a configuration file
    generated by running the sacctmgr dump command. This command does
    not load archive data, see the sacctmgr archive load option instead.
    .IP

    .TP
    \fBmodify\fR <\fIENTITY\fR> \fBwhere\fR <\fISPECS\fR> \fBset\fR <\fISPECS\fR>
    Modify an entity.
    .IP

    .TP
    \fBping\fR
    Ping slurmdbd.
    .IP

    .TP
    \fBreconfigure\fR
    Reconfigures the SlurmDBD if running with one.
    .IP

    .TP
    \fBremove\fR <\fIENTITY\fR> \fBwhere\fR <\fISPECS\fR>
    Delete the specified entities.
    Identical to the \fBdelete\fR command.
    .IP

    .TP
    \fBshow\fR <\fIENTITY\fR> [<\fISPECS\fR>]
    Display information about the specified entity.
    By default, all entries are displayed, you can narrow results by
    specifying SPECS in your query.
    Identical to the \fBlist\fR command.
    .IP

    .TP
    \fBshutdown\fR
    Shutdown the server.
    \fBexit\fP
    Terminate sacctmgr interactive mode.
    Identical to the \fBquit\fR command.
    .IP

    .TP
    \fBquiet\fP
    Print no messages other than error messages.
    .IP

    .TP
    \fBquit\fP
    Terminate the execution of sacctmgr interactive mode.
    Identical to the \fBexit\fR command.
    .IP

    .TP
    \fBverbose\fP
    Enable detailed logging.
    This includes time\-stamps on data structures, record counts, etc.
    This is an independent command with no options meant for use in
    interactive mode.
    .IP

    .TP
    \fB!!\fP
    Repeat the last command.
    .IP

    .SH "ENTITIES"

    .TP
    \fBaccount\fR
    An account, typically specified at job submit time using the
    \fB\-\-account=\fR option.
    These may be arranged in a hierarchical fashion, for example
    accounts 'chemistry' and 'physics' may be children of
    the account 'science'.
    The hierarchy may have an arbitrary depth.
    .IP

    .TP
    \fBassociation\fR
    The entity used to group information consisting of four parameters:
    \fBaccount\fR, \fBcluster\fR, \fBpartition\fR (optional), and \fBuser\fR.
    Used only with the \fBlist\fR or \fBshow\fR command. Add, modify, and
    delete should be done to a user, account or cluster entity, which will
    in turn update the underlying associations. Modification of attributes like
    limits is allowed for an association but not a modification of the four
    core attributes of an association. You cannot change the partition setting
    (or set one if it has not been set) for an existing association. Instead,
    you will need to create a new association with the partition included. You
    system configuration.
    .IP

    .TP
    \fBcoordinator\fR
    A special privileged user, usually an account manager, that can
    add users or sub\-accounts to the account they are coordinator over.
    This should be a trusted person since they can change limits on
    account and user associations, as well as cancel, requeue or reassign
    accounts of jobs inside their realm.
    .IP

    .TP
    \fBevent\fR
    Events like downed or drained nodes on clusters. Note that this does not
    include transitory states like DRAINING.
    .IP

    .TP
    \fBfederation\fR
    A group of clusters that work together to schedule jobs.
    .IP

    .TP
    \fBjob\fR
    Used to modify specific fields of a job: Derived Exit Code, Comment,
    AdminComment, Extra, SystemComment, TRES, or WCKey.
    .IP

    .TP
    \fBproblem\fR
    Use with \fBshow\fR or \fBlist\fR to display entity problems.
    .IP

    .TP
    \fBqos\fR
    Quality of Service.
    .IP

    .TP
    \fBreservation\fR
    A collection of resources set apart for use by a particular account, user
    or group of users for a given period of time.
    .IP

    .TP
    \fBresource\fR
    Software resources for the system. Those are software licenses shared
    among clusters.
    .IP

    .TP
    .TP
    \fBstats\fR
    Used with \fBlist\fR or \fBshow\fR command to view server statistics.
    Accepts optional argument of \fBave_time\fR or \fBtotal_time\fR to sort on those
    fields. By default, sorts on increasing RPC count field.
    .IP

    .TP
    \fBtransaction\fR
    List of transactions that have occurred during a given time period.
    .IP

    .TP
    \fBtres\fR
    Used with \fBlist\fR or \fBshow\fR command to view a list of Trackable
    RESources configured on the system.
    .IP

    .TP
    \fBuser\fR
    The login name. Usernames are case\-insensitive (forced to lowercase) unless
    the \fBPreserveCaseUser\fR option has been set in the SlurmDBD configuration
    file.
    .IP

    .TP
    \fBwckeys\fR
    Workload  Characterization  Key. An arbitrary  string  for  grouping orthogonal accounts.
    .IP

    .SH "GENERAL SPECIFICATIONS FOR ASSOCIATION BASED ENTITIES"
    \fBNOTE\fR: The group limits (GrpJobs, GrpTRES, etc.) are tested when a job is
    being considered for being allocated resources.
    If starting a job would cause any of its group limit to be exceeded,
    that job will not be considered for scheduling even if that job might preempt
    other jobs which would release sufficient group resources for the pending
    job to be initiated.

    \fBNOTE\fR: TRES limit modifications of any kind using "+=" or "\-=" syntax
    will always produce a value between 0 and 18446744073709551600.
    Trying to set a negative value using "\-=" will set that TRES to 0, even if
    there was not a TRES limit present beforehand.

    .TP
    \fBDefaultQOS\fR=<\fIdefault_qos\fR>
    The QOS this association and its children will use by default if allowed in the
    \fBQosLevel\fR list mentioned below.
    This is overridden if set directly on a user.
    To clear an existing value, set a new value of \-1.
    .IP

    .IP \fBFairshare\fR={<\fIfairshare_number\fR>|parent}
    Maximum number of running jobs in aggregate for this association and its
    children.
    To clear an existing value, set a new value of \-1.
    .IP

    .TP
    \fBGrpJobsAccrue\fR=<\fImax_jobs\fR>
    Maximum number of pending jobs in aggregate able to accrue age priority for this
    association and its children.
    To clear an existing value, set a new value of \-1.
    .IP

    .IP \fBGrpSubmit\fR=<\fImax_jobs\fR>
    .PD 0
    .IP \fBGrpSubmitJobs\fR=<\fImax_jobs\fR>
    .PD
    Maximum number of jobs in a pending or running state at any time in aggregate
    for this association and its children.
    To clear an existing value, set a new value of \-1.
    .IP

    .TP
    \fBGrpTRES\fR=TRES[+|\-]=<\fImax_TRES\fR>[,TRES[+|\-]=<\fImax_TRES\fR>,...]
    Maximum number of TRES able to be allocated by running jobs in aggregate for
    this association and its children.

    Refer to the \fBTRES information\fR section below for further details.
    .IP

    .TP
    \fBGrpTRESMins\fR=TRES[+|\-]=<\fIminutes\fR>[,TRES[+|\-]=<\fIminutes\fR>,...]
    Maximum number of TRES minutes that can possibly be used by past, present, and
    future jobs in this association and its children.

    Refer to the \fBTRES information\fR section below for further details.

    \fBNOTE\fR: This limit is not enforced if set on the root
    association of a cluster. So even though it may appear in sacctmgr
    output, it will not be enforced.

    \fBNOTE\fR: This limit only applies when using the Priority Multifactor
    plugin. The time is decayed using the value of PriorityDecayHalfLife
    or PriorityUsageResetPeriod as set in the slurm.conf. When this limit
    is reached all associated jobs running will be killed and all future
    jobs submitted with associations in the group will be delayed until
    they are able to run inside the limit.
    .IP

    .TP
    \fBGrpTRESRunMins\fR=TRES[+|\-]=<\fIminutes\fR>[,TRES[+|\-]=<\fIminutes\fR>,...]
    Maximum number of TRES minutes able to be allocated by running jobs in this
    association and its children. This takes into consideration time limit of

    \fBNOTE\fR: Although it may appear in sacctmgr output, this limit will not
    be enforced if set on the root association of a cluster.

    \fBNOTE\fR: This limit only applies when using the Priority Multifactor
    plugin. The time is decayed using the value of PriorityDecayHalfLife
    or PriorityUsageResetPeriod as set in the slurm.conf. When this limit
    is reached all associated jobs running will be killed and all future
    jobs submitted with associations in the group will be delayed until
    they are able to run inside the limit.
    .IP

    .TP
    \fBMaxJobs\fR=<\fImax_jobs\fR>
    Maximum number of running jobs per user in this association. This is overridden
    if set directly on a user. Default is the cluster's limit.
    To clear an existing value, set a new value of \-1.
    .IP

    .TP
    \fBMaxJobsAccrue\fR=<\fImax_jobs\fR>
    Maximum number of pending jobs able to accrue age priority at any given time in
    this association. This is overridden if set directly on a user.
    Default is the cluster's limit.
    To clear an existing value, set a new value of \-1.
    .IP

    .IP \fBMaxSubmit\fR=<\fImax_jobs\fR>
    .PD 0
    .IP \fBMaxSubmitJobs\fR=<\fImax_jobs\fR>
    .PD
    Maximum number of jobs in a pending or running state at any time in this
    association. Default is the cluster's limit.
    To clear an existing value, set a new value of \-1.
    .IP

    .IP \fBMaxTRES\fR=TRES[+|\-]=<\fImax_TRES\fR>[,TRES[+|\-]=<\fImax_TRES\fR>,...]
    .PD 0
    .IP \fBMaxTRESPJ\fR=TRES[+|\-]=<\fImax_TRES\fR>[,TRES[+|\-]=<\fImax_TRES\fR>,...]
    .PD 0
    .IP \fBMaxTRESPerJob\fR=TRES[+|\-]=<\fImax_TRES\fR>[,TRES[+|\-]=<\fImax_TRES\fR>,...]
    .PD
    Maximum number of TRES each job can use in this association.
    This is overridden if set directly on a user.
    Default is the cluster's limit.

    Refer to the \fBTRES information\fR section below for further details.
    .IP

    .IP \fBMaxTRESMins\fR=TRES[+|\-]=<\fIminutes\fR>[,TRES[+|\-]=<\fIminutes\fR>,...]
    .PD 0
    .IP \fBMaxTRESMinsPJ\fR=TRES[+|\-]=<\fIminutes\fR>[,TRES[+|\-]=<\fIminutes\fR>,...]
    Maximum number of TRES each node in a job allocation can use in this association.
    This is overridden if set directly on a user.
    Default is the cluster's limit.

    Refer to the \fBTRES information\fR section below for further details.
    .IP

    .IP \fBMaxWall\fR=<\fImax_wall\fR>
    .PD 0
    .IP \fBMaxWallDurationPerJob\fR=<\fImax_wall\fR>
    .PD
    Maximum wall clock time each job can use in this association.
    This is overridden if set directly on a user.
    Default is the cluster's limit.
    MaxWall format is <min> or <min>:<sec> or <hr>:<min>:<sec> or
    fR(5) man page for a full list of flags. The environment
    variable takes precedence over the setting in the slurm.conf.
    .IP

    .TP
    \fBSLURM_JSON\fR
    Control JSON serialization:
    .IP
    .RS
    .TP
    \fBcompact\fR
    Output JSON as compact as possible.
    .IP

    .TP
    \fBpretty\fR
    Output JSON in pretty format to make it more readable.
    .IP
    .RE

    .TP
    \fBSLURM_YAML\fR
    Control YAML serialization:
    .IP
    .RS
    .TP
    \fBcompact\fR Output YAML as compact as possible.
    .IP

    $ sacctmgr create account name=physics parent=science fairshare=20
    $ sacctmgr create user name=adam cluster=tux account=physics fairshare=10
    $ sacctmgr delete user name=adam cluster=tux account=physics
    $ sacctmgr delete user name=adam cluster=tux account=science partition=\\"\\"
    $ sacctmgr delete account name=physics cluster=tux
    $ sacctmgr modify user where name=adam cluster=tux account=physics set maxjobs=2 maxwall=30:00
    $ sacctmgr add user brian account=chemistry
    $ sacctmgr list associations cluster=tux format=Account,Cluster,User,Fairshare tree withd
    $ sacctmgr list transactions Action="Add Users" Start=11/03\-10:30:00 format=Where,Time
    $ sacctmgr dump cluster=tux file=tux_data_file
    $ sacctmgr load tux_data_file
    .fi

    A user's account can not be changed directly. A new association needs to be
    created for the user with the new account. Then the association with the old
    account can be deleted.

    When modifying an object placing the key words 'set' and the
    optional 'where' is critical to perform correctly below are examples to
    produce correct results. As a rule of thumb anything you put in front
    of the set will be used as a quantifier. If you want to put a
    quantifier after the key word 'set' you should use the key
    word 'where'. The following is wrong:

    .nf
    $ sacctmgr modify user name=adam set fairshare=10 cluster=tux
    .fi

    This will produce an error as the above line reads modify user adam
    set fairshare=10 and cluster=tux. Either of the following is correct:

    .nf
    $ sacctmgr modify user name=adam cluster=tux set fairshare=10
    $ sacctmgr modify user name=adam set fairshare=10 where cluster=tux
    .fi

    When changing qos for something only use the '=' operator when wanting
    to explicitly set the qos to something. In most cases you will want
    to use the '+=' or '\-=' operator to either add to or remove from the
    existing qos already in place.

    If a user already has qos of normal,standby for a parent or it was
    explicitly set you should use qos+=expedite to add this to the list in
    this fashion.

    If you are looking to only add the qos expedite to only a certain
    account and or cluster you can do that by specifying them in the
    sacctmgr line.

    .nf
    $ sacctmgr modify user name=adam set qos+=expedite
    .fi
         normal
       expedite
    .fi

    List all the associations in the cluster.

    .nf
    $ sacctmgr show assoc format=cluster,account,qos
       Cluster     Account                  QOS
      \-\-\-\-\-\-\-\-  \-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
         zebra        root               normal
         zebra        root               normal
         zebra           g               normal
         zebra          g1               normal
    .fi

    Add the QOS expedite to account G1 and display the result.
    Using the operator += the QOS will be added together
    with the  existing QOS to this account.

    .nf
    $ sacctmgr modify account name=g1 set qos+=expedite
    $ sacctmgr show assoc format=cluster,account,qos
       Cluster     Account                  QOS
      \-\-\-\-\-\-\-\-  \-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
         zebra        root               normal
         zebra        root               normal
         zebra           g               normal
         zebra          g1      expedite,normal
    .fi

    Now set the QOS expedite as the only QOS for the account G and display
    the result. Using the operator = that expedite is the only usable
    QOS by account G

    .nf
    $ sacctmgr modify account name=G set qos=expedite
    $ sacctmgr show assoc format=cluster,account,qos
       Cluster     Account                  QOS
      \-\-\-\-\-\-\-\-  \-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-
         zebra        root               normal
         zebra        root               normal
         zebra           g             expedite
         zebra          g1      expedite,normal
    .fi

    If a new account is added under the account G it will inherit the
    QOS expedite and it will not have access to QOS normal.

    .nf
    $ sacctmgr add account banana parent=G
    $ sacctmgr show assoc format=cluster,account,qos
    \-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\- \-\-\-\-\-\-\-\-
           cpu                          1
           mem                          2
        energy                          3
          node                          4
       billing                          5
          gres         gpu:tesla     1001
       license               vcs     1002
            bb              cray     1003
    .fi

    .SH "COPYING"
    Copyright (C) 2008\-2010 Lawrence Livermore National Security.
    Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER).
    .br
    Copyright (C) 2010\-2022 SchedMD LLC.
    .LP
    This file is part of Slurm, a resource management program.
    For details, see <https://slurm.schedmd.com/>.
    .LP
    Slurm is free software; you can redistribute it and/or modify it under
    the terms of the GNU General Public License as published by the Free
    Software Foundation; either version 2 of the License, or (at your option)
    any later version.
    .LP
    Slurm is distributed in the hope that it will be useful, but WITHOUT ANY
    WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
    FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
    details.

    .SH "SEE ALSO"
    \fBslurm.conf\fR(5),
    \fBslurmdbd\fR(8)