Manual Pages
Table of Contents
na_nfsstat - display NFS statistics
nfsstat interval [ count ]
nfsstat -h [ -n [ count ] ] | [ [ ip_address | host_name ]
[ interval ] ]
nfsstat -l [ count ]
nfsstat -z
nfsstat -d
nfsstat -C
nfsstat [ -c ] [ -t ]
nfsstat displays statistical information about NFS (Network
File System) and RPC (Remote Procedure Call) for the
filer. That includes statistics for NFS versions 2, 3 and
4. It can also be used to reinitialize this information.
If no arguments are given, nfsstat displays statistical
information since last zeroed with the -z option (or since
reboot if statistics have not been zeroed).
If just the interval argument is specified, nfsstat continuously
displays the summary information for the following
NFS requests: getattr, lookup, readlink, read, write,
create, remove, and readdir/readdirplus. The output will
show incremental statistics for the interval (in seconds)
since the command was executed. If count argument is also
specified, then the command exits after displaying the
statistics for the specified number of intervals. The
interval specified must be an integer.
Per-client statistics can also be collected and displayed
by enabling the nfs.per_client_stats.enable options (using
the options command - see na_options(1)) and invoking nfsstat
with the -h or the -l options. Per-client statistics
can be collected for up to the first 1000 NFS clients,
that have mounted the filesystem on the given filer. These
statistics are displayed in the decreasing order of the
total NFS ops performed by each client.
-
-h
- Displays per-client statistics since last zeroed
with the -z option (or since reboot if statistics
have not been zeroed). The statistics are displayed
on a per-client basis, with the IP address
and host name (where available) of each client
being displayed at the head of each client's block
of statistics.
If optional count is specified using -n option,
then per-client statistics for that many clients
are displayed.
If an optional IP address or host name is specified
with the -h option, only the statistics associated
with this client are displayed.
Statistics for read and write sizes are also displayed
as number of read or write requests made by
each client grouped by size of request. Without
this option, the cumulative statistics for all
clients are displayed.
If the IP address or host name is specified, an
optional interval argument can be supplied. This
will cause interval statistics to be displayed for
the specified client. The behavior is identical as
described in the Description paragraph.
-
-l
- Displays a list of the clients whose statistics
have been collected on a per-client basis, along
with the total NFS calls for that client since last
reboot, or last zeroed with the -z option, or last
explicit user request to enable the per-client
statistics, the count being displayed both as the
actual count and as a percentage of calls from all
clients. The list is sorted in the decreasing order
of the total calls for each client. If count argument
is specified, then the command exits after
displaying the statistics for the first count number
of clients and if it is not specified the
statistics for up to the first 256 clients are displayed,
ordered by the total NFS ops performed by
each client.
-
-z
- Zeroes (reinitializes) the current cumulative and
per client statistics. (However, statistics since
boot are retained.) The per clients statistics also
get reinitialized when nfs.per_client_stats.enable
option is turned off.
-
-c
- Includes reply cache statistics in the data displayed.
The reply cache is used to cache replies to
non-idempotent nfs requests, so that in case of a
retransmit, the correct reply can be sent back to
the client. Non-idempotent operations do not return
same results if repeated e.g. remove, create. The
various fields displayed are:
-
In Progress
- Number of requests which were
being processed at the time
retransmit requests ware
received for them.
-
Delay Hits
- Number of requests which were
dropped because they came in
within nfs_rc_delay of sending
out the reply. This field is
deprecated. A zero value will
always be printed for this
field. This is to maintain
backward compatibility with
previous ONTAP releases.
-
Misses
- Number of requests that were
not found in the reply cache.
These also include requests
coming in for the first time.
-
Idempotent
- Number of idempotent requests
saved in the reply cache for
the purpose of improving performance.
Non-idempotent Number of non-idempotent
requests replied to from the
reply cache.
-
-t
- Displays the statistics since boot time, rather
than since the last time they were zeroed.
-
-C
- Displays the number and type of NFS version 2 and 3
requests received by all FlexCache volumes. In
addition we display the number of each type that
resulted in a cache hit.
-
-d
- In addition to reply cache statistics, also
includes statistics about incoming messages and
allocated mbufs. The various fields displayed are:
-
nfs cache size
- NFS reply cache size. The cache
size depends on the Filer memory.
-
hash size
- NFS reply cache hash size.
-
num msg
- Number of incoming requests
(over UDP) dropped because
there are no free entries in
the NFS operation table.
-
too many mbufs
- Number of requests (over UDP)
dropped because they require
more mbufs than are available.
-
rpcErr
- Total number of rpc errors
encountered while generating
the rpc reply header.
-
svrErr
- Total number of RPC errors
encountered while processing
the request.
-
msg queued
- Total number of NFS connections
(over TCP) that have been
deferred waiting for NFS
resources to become available.
The NFS requests queued on
these connections will be processed
once these resources
become available. Connections
can get queued up when the
filesystem is busy doing back
to back CPs.
no msg re-queued(xfc)
Total number of NFS connections
(over TCP) which are in transmit
flow control and for which
further request processing has
been deferred because there are
too many replies queued on the
socket waiting to be acknowledged
by the client.
-
no msg dropped
- Number of incoming requests
(over UDP) dropped because
there are no free entries
available in the NFS operation
table. Its the same as num msg.
Requests can be dropped when
the filesystem is busy doing
back to back CPs.
-
no msg unqueued
- Number of connections unqueued
from the deferred list.
-
no msg discarded
- Number of deferred connections,
waiting for NFS resources to
become available, dropped
because of a timeout or shutdown
of the socket.
no msg dropped from vol offline
Number of requests dropped
because the volume was offline.
no deferred msg processed
Number of NFS requests unqueued
and processed from NFS over TCP
connections that had been put
on a queue waiting for NFS
resources to become available.
This is not the same as number
of NFS connections queued up
waiting for resources. Note
that one connection can contain
more than one NFS requests
waiting to be processed.
-
sbfull queued
- Number of requests queued
because the send buffer is
full.
-
sbfull unqueued
- Number of requests unqueued
when space available in the
send buffer.
-
sbfull discarded
- Number of requests waiting on a
full send buffer dropped
because of a forced shutdown of
the socket.
-
no mbuf queued
- Number of requests (over TCP)
waiting for mbufs to send a
reply.
-
no mbuf dropped
- Number of requests (over UDP)
dropped because of unavailability
of enough mbufs to send a
reply.
-
no mbuf unqueued
- Number of requests (over TCP)
processed from the wait queue
when enough mbufs become available.
no mbuf discarded Number of requests (over TCP)
waiting for mbufs dropped
because nothing can be sent
over the socket. This can
happen if the socket is closing.
(cumulative) active Two values are printed here.
The maximum (cumulative) number
of entries available in the NFS
operation table and the total
number of entries processing
requests at that time.
-
req mbufs
- Number of mbufs allocated for
an incoming NFS request over
UDP.
tcp input flowcontrol receive
Number of times the Filer went
into receive flow control.
tcp input flowcontrol xmit
Number of times the Filer went
into transmit flow control.
tcp input flowcontrol out receive
Number of times the Filer came
out of receive flow control.
tcp input flowcontrol out xmit
Number of times the Filer came
out of transmit flow control.
sockets zapped nfs Number of nfs sockets disconnected
because the send upcall
argument in the underlying tcp
socket had been zeroed.
sockets zapped tcp Number of tcp sockets disconnected
because both the send
and receive upcall arguments in
that tcp socket had been
zeroed.
-
no delegation
- Number of times no delegation
was handed out due to some
error.
-
read delegation
- Number of read delegations
handed out to NFS version 4
clients.
-
write delegation
- Number of write delegations
handed out to NFS version 4
clients.
-
v4 acls set
- Total number of acls set on
files/dir through an explicit
setattr command or when a file
or dir is created and an acl is
supplied in the initial
attributes.
-
nfs msgs counts
- The tot is the total number of
available NFS messages. The
unallocated is the number of
messages which have yet to be
allocated from the system. The
free is the number of allocated
messages which are not in use.
The used is the number of allocated
messages which are currently
in use. The "VM cb
heard" is how many times virtual
memory pressure has asked
for messages to be released
back to the system. The "VM cb
done" is how many messages were
released back to the system.
nfs reply cache counts
The tot is the total number of
available reply cache entries.
The unallocated is the number
of entries which have yet to be
allocated from the system. The
free is the number of allocated
entries which are not in use.
The used is the number of allocated
entries which are currently
in use. The "VM cb
heard" is how many times virtual
memory pressure has asked
for entries to be released back
to the system. The "VM cb done"
is how many entries were
released back to the system.
v2 mount (requested, granted, denied, resolving)
The requested is the number of
v2 mount requests. The granted
is the number of mount requests
granted access. The denied is
the number denied access. The
resolving is the number of
mount requests currently awaiting
access resolution.
v2 unmount (requested, granted, denied)
The requested is the number of
v2 unmount requests. The
granted is the number of
unmount requests granted. The
denied is the number denied.
v2 unmount all (requested, granted, denied)
The requested is the number of
v2 unmount all requests. The
granted is the number of
unmount all requests granted.
The denied is the number
denied.
v3 mount (requested, granted, denied, resolving)
The requested is the number of
v3 mount requests. The granted
is the number of mount requests
granted access. The denied is
the number denied access. The
resolving is the number of
mount requests currently awaiting
access resolution.
v3 unmount (requested, granted, denied)
The requested is the number of
v3 unmount requests. The
granted is the number of
unmount requests granted. The
denied is the number denied.
v3 unmount all (requested, granted, denied)
The requested is the number of
v3 unmount all requests. The
granted is the number of
unmount all requests granted.
The denied is the number
denied.
mount service requests (curr, total, max, redriven)
These are the requests made to
the mount service. These
include requests for mount,
umount, showmount, etc. The
curr is the number of current
requests to the mount service.
The total is the total number
of requests made to the mount
service. The max is the high
water mark of the curr counter.
The redriven is the total number
of mount service requests
that needed to be redriven
after an access cache population.
access cache (hits, partial misses, misses)
A request coming into the cache
can be looking for any combination
of read, write and root
attributes. The hits are the
number of requests which were
serviced completely by an entry
already in the access cache.
The partial misses are the number
of requests that were not
serviced completely by an entry
in the access cache. The misses
are the number of requests
which were not serviced by the
access cache because there was
no entry in the cache for that
client. Note that in general,
the ratio hits to the sum of
hits and partial misses +
misses should be high. I.e.,
we want significantly more hits
than misses.
access cache lookup requests (curr, total, max)
The curr is the number of current
requests which are being
serviced. This number is also
reprentative of the number of
entries on the export worker
queue. The total is the sum of
all requests. The max is the
high water mark of the curr
counter.
access cache nodes(found, created)
The found is the number of
entries that were hit as a
result of a cache lookup. The
created is the total number of
access cache nodes created
since boot.
access cache requests(queued, unqueued)
The queued is the number of
requests that had to wait (and
were queued) for the access
cache to be populated before
they could be processed. The
unqueued is the number of
requests that were unqueued and
processed after the access
cache was populated.
access cache requests unqueued by (flush, restore)
These are the number of
requests waiting for access
cache population which were
restarted when that access
cache was flushed through the
exportfs -f command or was
restored from disk upon reboot
or failover.
access cache read requests (queued, unqueued)
The queued counter is for
requests which got a cache miss
for read access and had to wait
because that information was
not in the cache. The unqueued
counter is for requests which
were restarted once the read
access information was inserted
in the access cache.
access cache write requests (queued, unqueued)
The queued counter is for
requests which got a cache miss
for write access and had to
wait because that information
was not in the cache. The
unqueued counter is for
requests which were restarted
once the write access information
was inserted in the access
cache.
access cache root requests (queued, unqueued)
The queued counter is for
requests which got a cache miss
for root access and had to wait
because that information was
not in the cache. The unqueued
counter is for requests which
were restarted once the root
access information was inserted
in the access cache.
access cache expired hits (total, read, write,
root)
These are the number of hits in
the access cache for read,
write or root attribute where
the access information for that
attribute had not been
refreshed for sometime. The
attribute information expires
based on the timeout value set
in nfs.export.neg.timeout and
nfs.export.pos.timeout.
access cache inserts (full, partial, dup, subnet,
restore)
The full count represents the
number of inserts into the
access cache where all
attributes upon which one or
more requests were waiting,
were resolved. As a result of a
full insert one or more
requests would be restarted.
The partial count represents
the number of inserts where
some of the attributes upon
which one or more requests were
waiting, were resolved. No
requests are restarted as a
result of a partial insert.
The dup count represents the
number of times an insert was
done for an attribute when the
access cache already contained
some information about that
attribute. However its not neccessary
that the same access
information was inserted the
second time around. e.g. If
the cache contained information
about node A.B.C.D as read
being allowed and then another
insert was done in the cache
which changed this information
to read being denied for
A.B.C.D, this would be considered
a dup insert. The subet
count represents the number of
times access information was
inserted for a subnet. The
restore count represents the
number of times an insert was
done based on information on
disk. This happens upon reboot
or cluster failover.
access cache refreshes requested (total, read,
write, root)
These counts represent for each
attribute the number of times
stale information was returned
by the cache and a refresh was
requested for that attribute.
access cache refreshes done (total, read, write,
root)
These counts represent for each
attribute the number of times a
refresh was done for that
attribute.
access cache errors (query, insert, no mem)
The query counter indicates the
total number of errors (including
no mem) encountered when
doing a query in the cache.
The insert counter indicates
the total number of errors
(including no mem) encountered
when doing an insert in the
cache. The no mem counter
indicates the number of times a
query or insert could not be
done because memory was not
available.
access cache nodes (flushed, harvested, harvests
failed)
The flushed count represents
the number of nodes removed
from the access cache as a
result of an exportfs -f command.
The harvested count represents
the number of nodes
removed from the access cache
which have not been accessed
for more than nfs.export.harvest.timeout
seconds. The harvests
failed count represents
the number of times a node was
found in a state where requests
were waiting for resolution on
it and that node had not been
accessed for more than
nfs.export.harvest.timeout seconds.
High value of this
counter may point to issues
with name resolution service.
access cache nodes (allocated, free)
The allocated count represents
the total number of nodes allocated
by all the access caches.
The free count represents the
number of nodes which are not
in use by any access cache.
access cache qctx (allocated, free)
The allocated count represents
the total number of query contexts
allocated to queue
requests waiting for resolution.
The free count represents
the number of contexts
which are not in use.
access cache persistence errors(total)
The total count represents the
errors encountered while saving
or restoring access cache data
to/from disk.
access cache persistence nodes handled
(restored,saved)
The restored count represents
the number of nodes that were
restored from the persistent
access cache. The saved count
represents the number of nodes
saved since the last system
boot. This is cumulative and
periodically increases after
periodic saving of access cache
into persistent storage.
access cache persistence rules deleted (total)
The total count represents the
number of times we reference a
rule which no longer exists
while restoring content from
persistent access cache. Contents
for such rules cannot
(and need not) be restored.
access cache persistence memchunks (allocated,
freed)
The allocated count represents
the number of memory chunks
allocated for use while encoding
access cache data for persistent
storage. The freed represents
the number of memory
chunks freed (after having been
allocated) while processing
access cache data for persistent
storage.
assist queue (queued, split mbufs, drop for EAGAIN)
The queued is the number of NFS
requests which are put on the
assist queue. The "split mbufs"
is the number of NFS requests
in which data spans multiple
mbufs. The "drop for EAGAIN" is
the number of NFS requests
which were put on the assist
queue for servicing an access
cache miss and for which name
service errors resulted in the
request being dropped.
NFS re-drive queue(curr, max, total)
The curr is the number of NFS
requests currently on the redrive
queue, waiting to be processed.
The max is the maximum
number of NFS requests that
were waiting on the re-drive
queue at any given time. The
total is the total number of
NFS requests processed so far.
Direct NFS re-drive(memory, webNFS)
Shows the number of NFS
requests directly queued from
the driver thread to the redrive
queue. The memory is the
number of the NFS requests
queued up to the re-drive queue
because of the failure of the
non-blocking memory allocation.
The webNFS is the number of
webNFS requests put on the redrive
queue.
Errors in the blocking export access check
Shows the count of the errors
received during blocking export
access check.
RPCSEC_GSS context limit
The limit on the RPCSEC_GSS
context table. This limit is
defined via the nfs.rpcsec.ctx.high
- option.
- RPCSEC_GSS
refers to the authentication
protocol used by NFS
(see RFC2203) for encapsulating
Kerberos V5 authentication
information. Therefore an RPCSEC_GSS
context corresponds to
a Kerberos V5 authentication
session between the NFS client
and server.
current context count
The number of RPCSEC_GSS contexts
currently allocated.
maximum context count
The maximum number of RPCSEC_GSS
contexts ever allocated
since the last boot. Note that
this value is not zeroed when
the -z option is used.
context reclaim callbacks
context idle/expired scans
vm pressure callbacks
Each of these three statistics
counts an asynchronous attempt
(callback) to reclaim least
recently used contexts from the
RPCSEC_GSS context table.
The "context reclaim callbacks"
statistic counts the number of
callbacks triggered by the RPCSEC_GSS
subsystem, due to a
deferred reclaim (because ONTAP
could not suspend).
The "context idle/expired
scans" statistic counts the
number of periodic scans to
look for expired or idle contexts.
A context is considered
idle if it has not been used
for the number of seconds set
in the nfs.rpcsec.ctx.idle
option.
The "vm pressure callbacks"
statistic counts the number of
times the ONTAP VM subsystem
asked the RPCSEC_GSS subsystem
to release memory back to the
system free list of memory.
-
contexts created
- Counts the total number of RPCSEC_GSS
contexts created.
-
contexts deleted
- Counts the total number of RPCSEC_GSS
contexts deleted.
contexts deleted due to vm pressure
Counts the RPCSEC_GSS contexts
deleted due to vm pressure
callbacks.
contexts deleted due to context limit
Counts the RPCSEC_GSS contexts
deleted due to the current context
count already being at the
context limit whenever a new
context needs to be created.
The least recently used context
will be deleted.
contexts deleted due to idle/expiration
Counts the number of RPCSEC_GSS
contexts deleted because the
expired or were idle.
requests exceeding timeout
The number of NFS requests
which exceeded the timer set in
the option nfs.response.trig_ger.
The server RPC display includes the following fields, with
separate values for TCP and UDP:
-
calls
- The total number of RPC calls received.
badcalls The total number of calls rejected by the
RPC layer (the sum of badlen and xdrcall
as defined below).
nullrecv The number of times an RPC call was not
available when it was thought to be
received.
-
badlen
- The number of RPC calls with a length
shorter than a minimum-sized RPC call.
Also counts the number of RPC requests
over the TCP transport with illegal
record lengths, such as those that are
too small, too large, or not a multiple
of the base XDR encoding unit (4 bytes).
xdrcall The number of RPC calls whose header
could not be XDR decoded.
The server NFS display shows the number of NFS calls
received (calls) and rejected (badcalls), and the counts
and percentages for the various calls that were made.
An NFSV4 compound request can contain more than one operations.
Therefore, the nfsstat command displays the total
number of rpc requests received by the server (calls) as
well as the total operations (ops) received in all the
compound requests.
Only a count is displayed for NULL and COMPOUND requests.
Percentages are displayed for all operations received in
COMPOUND requests relative to the total number of operations.
The total calls displayed under the TCP, UDP and
Server nfs headings represents the sum of all RPC requests
received from NFSV2, NFSV3 and NFSV4 clients and do not
take into account any operations in COMPOUND requests.
nfsstat -l reports unexpected percentages, if the
nfs.per_client_stats.enable option is enabled after the
system has been running for a while (typically this option
should be enabled at system startup time via the /etc/rc
file). Resetting the statistics via the nfsstat -z command
will clear this condition.
In takeover mode, the nfsstat command displays combined
statistics for the live filer and the failed filer. From
the statistics, you cannot determine how many requests
were serviced by the live filer and how many requests were
serviced on behalf of the failed filer.
The -h and -l options display the combined client information
for both the live filer and the failed filer.
The NFS statistics are cumulative. That is, a giveback
operation does not zero out the NFS statistics. After giving
back the failed filer's resources, the live filer does
not subtract the statistics about NFS operations it performed
on behalf of the failed filer when it was in
takeover mode.
When run from a vfiler context, (e.g. via the vfiler run
command), nfsstat operates on the concerned vfiler. Note
that if vfilers are licensed when using the -h and -l
options it is necessary to run this command from a vfiler
context, since the interpretations of the hosts are dependent
on the vfiler.
na_netstat(1), na_options(1), na_partner(1), na_vfiler(1),
na_sysstat(1).
Table of Contents