Different Ways to Check Memory Usage on Solaris Server


To get one of our new Solaris servers well optimized, I’ve tried different ways to check memory usage recently. Here, I would like to briefly list them for reference in different sections.

Unix Commands

1.  echo ::memstat | mdb –k

# echo ::memstat | mdb -k

Page Summary                Pages                MB  %Tot

------------     ----------------  ----------------  ----

Kernel                     494128              3860   12%

ZFS File Data              636113              4969   16%

Anon                      1825944             14265   45%

Exec and libs               55216               431    1%

Page cache                 142520              1113    3%

Free (cachelist)           694114              5422   17%

Free (freelist)            241821              1889    6%

Total                     4089856             31952

Physical                  4072965             31820

Where,

Kernel: Kernel page

Anon: anonymous pages (such as stack, heap, shared mem etc)

Exec and libs: executables and libraries

Page cache:  file cache

Free (cachelist) + Free (freelist) = freemem(value for column “free” when “vmstat” is issued)

This command is very helpful if you want to have a big picture of memory usage on your unix server. For instance, we can generally know how much memory has been consumed by ZFS and Kernel processes.

2. prstat –t

Instead of command “prstat”, which report active process statistics, command prstat with opetion “-t” can report total usage summary for each user. The summary includes the total number of processes or LWPs owned by the user, total size of process images, total resident set size, total cpu time, and percentages of recent cpu time and system memory.

# prstat -t

 NPROC USERNAME  SWAP   RSS MEMORY      TIME  CPU

   377 oracle     16G   16G    50% 133:03:55 0.1%

    37 root      198M  238M   0.7%   3:48:10 0.0%

     1 noaccess   82M  163M   0.5%   3:07:53 0.0%

     1 smmsp    1488K 9080K   0.0%   0:00:41 0.0%

     2 daemon   2280K 8456K   0.0%   0:00:32 0.0%

Total: 418 processes, 6821 lwps, load averages: 0.65, 0.53, 0.59

3. ps -efo pmem,uid,pid,ppid,pcpu,comm | sort -r

This command can present the list of processes by sorting memory usage. It’s kindly helpful if you need such information.

# ps -efo pmem,uid,pid,ppid,pcpu,comm | sort -r

11.1   101 24515     1  0.0 ora_arc3_ORCL

11.1   101 24513     1  0.0 ora_arc2_ORCL

11.0   101 28509     1  0.0 ora_cjq0_ORCL

11.0   101 27673     1  0.0 oracleORCL

11.0   101 27671     1  0.0 oracleORCL

11.0   101 27669     1  0.0 oracleORCL

11.0   101 26911     1  0.0 oracleORCL

11.0   101 26868     1  0.0 oracleORCL

11.0   101 26866     1  0.0 oracleORCL

11.0   101 26852     1  0.0 oracleORCL

11.0   101 26850     1  0.0 oracleORCL

11.0   101 26833     1  0.0 oracleORCL

11.0   101 26827     1  0.0 oracleORCL

11.0   101 26799     1  0.0 oracleORCL

11.0   101 26788     1  0.0 oracleORCL

11.0   101 24585     1  0.0 ora_q001_ORCL

11.0   101 24537     1  0.0 ora_q000_ORCL

11.0   101 24523     1  0.0 ora_qmnc_ORCL

11.0   101 24519     1  0.0 ora_fbda_ORCL

11.0   101 24511     1  0.0 ora_arc1_ORCL

11.0   101 24509     1  0.0 ora_arc0_ORCL

11.0   101 24491     1  0.0 ora_s000_ORCL

11.0   101 24489     1  0.0 ora_d000_ORCL

11.0   101 24487     1  0.0 ora_mmnl_ORCL

11.0   101 24485     1  0.0 ora_mmon_ORCL

11.0   101 24483     1  0.0 ora_reco_ORCL

11.0   101 24481     1  0.0 ora_smon_ORCL

11.0   101 24479     1  0.0 ora_ckpt_ORCL

11.0   101 24477     1  0.0 ora_lgwr_ORCL

11.0   101 24475     1  0.0 ora_dbwf_ORCL

11.0   101 24473     1  0.0 ora_dbwe_ORCL

11.0   101 24471     1  0.0 ora_dbwd_ORCL

11.0   101 24469     1  0.0 ora_dbwc_ORCL

11.0   101 24467     1  0.0 ora_dbwb_ORCL

11.0   101 24465     1  0.0 ora_dbwa_ORCL

11.0   101 24463     1  0.0 ora_dbw9_ORCL

11.0   101 24461     1  0.0 ora_dbw8_ORCL

11.0   101 24459     1  0.0 ora_dbw7_ORCL

11.0   101 24457     1  0.0 ora_dbw6_ORCL

11.0   101 24455     1  0.0 ora_dbw5_ORCL

11.0   101 24453     1  0.0 ora_dbw4_ORCL

Alternatively, the following command can be used too to collect similar information.

ps -eo pid,pmem,vsz,rss,comm | sort -rnk2 | head

4. /usr/proc/bin/pmap -x <process-id>

Processes have private memory to hold their stack space, heap, and data areas. The only way to see how much memory a process is actively using is to use above command. This command is available in Solaris 2.6  and later releases.

# /usr/proc/bin/pmap -x 8833

8833:   oracleORCL (LOCAL=NO)

         Address     Kbytes        RSS       Anon     Locked Mode   Mapped File

0000000100000000     103712     103312          -          - r-x--  oracle

0000000106646000        832        816        320          - rwx--  oracle

0000000106716000         16         16          8          - rwx--  oracle

000000010671A000         24         24         24          - rwx--    [ heap ]

0000000106720000       3776       2816       1536          - rwx--    [ heap ]

0000000380000000    1343488    1343488          -    1343488 rwxsR    [ ism shmid=0x3e ]

0000000400000000     802816     802816          -     802816 rwxsR    [ ism shmid=0x3f ]

0000000440000000     671744     671744          -     671744 rwxsR    [ ism shmid=0x40 ]

0000000480000000          8          8          -          8 rwxsR    [ ism shmid=0x41 ]

FFFFFFFF7A60E000          8          8          8          - rw--R    [ stack tid=11 ]

FFFFFFFF7A70E000          8          8          -          - rw--R    [ stack tid=10 ]

FFFFFFFF7A80E000          8          8          8          - rw--R    [ stack tid=9 ]

FFFFFFFF7A90E000          8          8          -          - rw--R    [ stack tid=8 ]

FFFFFFFF7AA0E000          8          8          -          - rw--R    [ stack tid=3 ]

FFFFFFFF7AF0E000          8          8          -          - rw--R    [ stack tid=2 ]

FFFFFFFF7B000000         64         64          -          - rw---    [ anon ]

FFFFFFFF7B100000         64         64         64          - rw---    [ anon ]

FFFFFFFF7B350000         64         64         64          - rw---    [ anon ]

FFFFFFFF7B40E000          8          8          8          - rw--R    [ stack tid=7 ]

FFFFFFFF7B4F0000         64         64         64          - rw---    [ anon ]

FFFFFFFF7B500000         64         64         64          - rw---    [ anon ]

FFFFFFFF7B510000         64         64         64          - rw---    [ anon ]

FFFFFFFF7B520000         64         64         64          - rw---    [ anon ]

FFFFFFFF7B530000        128        128        128          - rw---    [ anon ]

FFFFFFFF7B550000         64         64         64          - rw---    [ anon ]

FFFFFFFF7B560000        128        128        128          - rw---    [ anon ]

FFFFFFFF7B580000         64         64         64          - rw---    [ anon ]

FFFFFFFF7B590000         64         64         64          - rw---    [ anon ]

FFFFFFFF7B5A0000        128        128        128          - rw---    [ anon ]

FFFFFFFF7B5C0000        128        128        128          - rw---    [ anon ]

FFFFFFFF7B5E0000        128        128        128          - rw---    [ anon ]

FFFFFFFF7B600000        128        128        128          - rw---    [ anon ]

FFFFFFFF7B620000         64         64         64          - rw---    [ anon ]

FFFFFFFF7B630000         64         64         64          - rw---    [ anon ]

FFFFFFFF7B640000        128        128        128          - rw---    [ anon ]

FFFFFFFF7B660000        128        128        128          - rw---    [ anon ]

FFFFFFFF7B680000        128        128        128          - rw---    [ anon ]

FFFFFFFF7B6A0000        128        128        128          - rw---    [ anon ]

FFFFFFFF7B6C0000         64         64         64          - rw---    [ anon ]

FFFFFFFF7B70E000          8          8          -          - rw--R    [ stack tid=6 ]

FFFFFFFF7B80E000          8          8          8          - rw--R    [ stack tid=5 ]

FFFFFFFF7B90E000          8          8          -          - rw--R    [ stack tid=4 ]

FFFFFFFF7B9F0000         64         64         64          - rw---    [ anon ]

FFFFFFFF7BA00000         64         64         64          - rw---    [ anon ]

FFFFFFFF7BA10000         64         64         64          - rw---    [ anon ]

FFFFFFFF7BA20000        128        128        128          - rw---    [ anon ]

FFFFFFFF7BA40000        128        128        128          - rw---    [ anon ]

FFFFFFFF7BA60000        128        128        128          - rw---    [ anon ]

FFFFFFFF7BA80000        128        128        128          - rw---    [ anon ]

FFFFFFFF7BAA0000         64         64         64          - rw---    [ anon ]

FFFFFFFF7BAB0000        128        128        128          - rw---    [ anon ]

FFFFFFFF7BAD0000        128        128        128          - rw---    [ anon ]

FFFFFFFF7BB00000         16         16          -          - r-x--  liblgrp.so.1

FFFFFFFF7BC04000          8          8          8          - rwx--  liblgrp.so.1

FFFFFFFF7BD00000         56         56          -          - r-x--  libmd.so.1

FFFFFFFF7BE0E000          8          8          8          - rwx--  libmd.so.1

FFFFFFFF7BF00000        640        640          -          - r-x--  libm.so.2

FFFFFFFF7C000000         64         64          -          - rwx--    [ anon ]

FFFFFFFF7C09E000         40         40         16          - rwx--  libm.so.2

FFFFFFFF7C100000         32         32          -          - r-x--  librt.so.1

FFFFFFFF7C208000          8          8          -          - rwx--  librt.so.1

FFFFFFFF7C300000         32         32          -          - r-x--  libaio.so.1

FFFFFFFF7C408000          8          8          8          - rwx--  libaio.so.1

FFFFFFFF7C500000       1264       1176          -          - r-x--  libc.so.1

FFFFFFFF7C700000          8          8          8          - rwx--    [ anon ]

FFFFFFFF7C73C000         72         72         64          - rwx--  libc.so.1

FFFFFFFF7C74E000          8          8          8          - rwx--  libc.so.1

FFFFFFFF7C800000          8          8          -          - r-x--  libdl.so.1

FFFFFFFF7C902000          8          8          -          - rwx--  libdl.so.1

FFFFFFFF7CA00000         32         32          -          - r-x--  libgen.so.1

FFFFFFFF7CB08000          8          8          -          - rwx--  libgen.so.1

FFFFFFFF7CC00000         56         56          -          - r-x--  libsocket.so.1

FFFFFFFF7CD0E000         16         16          8          - rwx--  libsocket.so.1

FFFFFFFF7CE00000        688        680          -          - r-x--  libnsl.so.1

FFFFFFFF7CF00000         64         64         64          - rw---    [ anon ]

FFFFFFFF7CF10000         64         64         64          - rw---    [ anon ]

FFFFFFFF7CF20000         64         64         64          - rw---    [ anon ]

FFFFFFFF7CF30000         64         64         64          - rw---    [ anon ]

FFFFFFFF7CF40000         64         64         64          - rw---    [ anon ]

FFFFFFFF7CF50000         64         64         64          - rw---    [ anon ]

FFFFFFFF7CF60000         64         64         64          - rw---    [ anon ]

FFFFFFFF7CF70000         64         64         64          - rw---    [ anon ]

FFFFFFFF7CFAC000         64         64         16          - rwx--  libnsl.so.1

FFFFFFFF7CFBC000         32         32         24          - rwx--  libnsl.so.1

FFFFFFFF7D000000          8          8          -          - r-x--  libkstat.so.1

FFFFFFFF7D102000          8          8          8          - rwx--  libkstat.so.1

FFFFFFFF7D200000       2184       1256          -          - r-x--  libnnz10.so

FFFFFFFF7D500000         64         64         64          - rwx--    [ anon ]

FFFFFFFF7D520000        240        240        216          - rwx--  libnnz10.so

FFFFFFFF7D55C000          8          -          -          - rwx--  libnnz10.so

FFFFFFFF7D600000         72         72          -          - r-x--  libdbcfg10.so

FFFFFFFF7D700000         24         16          8          - rwx--    [ anon ]

FFFFFFFF7D710000          8          8          8          - rwx--  libdbcfg10.so

FFFFFFFF7D770000         64         64         64          - rw---    [ anon ]

FFFFFFFF7D780000         64         64         64          - rw---    [ anon ]

FFFFFFFF7D790000         64         64         64          - rw---    [ anon ]

FFFFFFFF7D7A0000         64         64         64          - rw---    [ anon ]

FFFFFFFF7D7B0000         64         64         64          - rw---    [ anon ]

FFFFFFFF7D7C0000         64         64         64          - rw---    [ anon ]

FFFFFFFF7D7D0000         64         64         64          - rw---    [ anon ]

FFFFFFFF7D7E0000         64         64         64          - rw---    [ anon ]

FFFFFFFF7D800000       9272       8264          -          - r-x--  libjox10.so

FFFFFFFF7E200000          8          8          8          - rwx--    [ anon ]

FFFFFFFF7E20C000        560        504        216          - rwx--  libjox10.so

FFFFFFFF7E300000       1072        728          -          - r-x--  libclsra10.so

FFFFFFFF7E500000          8          8          8          - rwx--    [ anon ]

FFFFFFFF7E50A000         56         40         24          - rwx--  libclsra10.so

FFFFFFFF7E518000          8          -          -          - rwx--  libclsra10.so

FFFFFFFF7E600000       1072        656          -          - r-x--  libocrutl10.so

FFFFFFFF7E800000          8          8          -          - r-x--  libc_psr.so.1

FFFFFFFF7E80A000         56         56          8          - rwx--  libocrutl10.so

FFFFFFFF7E818000          8          -          -          - rwx--  libocrutl10.so

FFFFFFFF7E900000       1376        856          -          - r-x--  libocrb10.so

FFFFFFFF7EB00000          8          8          8          - rwx--    [ anon ]

FFFFFFFF7EB56000         64         64         32          - rwx--  libocrb10.so

FFFFFFFF7EB66000          8          -          -          - rwx--  libocrb10.so

FFFFFFFF7EC00000       1536       1032          -          - r-x--  libocr10.so

FFFFFFFF7EE00000          8          8          8          - rwx--    [ anon ]

FFFFFFFF7EE7E000         64         56         32          - rwx--  libocr10.so

FFFFFFFF7EE8E000          8          -          -          - rwx--  libocr10.so

FFFFFFFF7EF00000          8          8          -          - r-x--  libskgxn2.so

FFFFFFFF7F000000          8          8          -          - rwx--  libskgxn2.so

FFFFFFFF7F100000       1752       1496          -          - r-x--  libhasgen10.so

FFFFFFFF7F300000          8          8          8          - rwx--    [ anon ]

FFFFFFFF7F3B4000         72         64         16          - rwx--  libhasgen10.so

FFFFFFFF7F3C6000         16          8          8          - rwx--  libhasgen10.so

FFFFFFFF7F400000        144        136          -          - r-x--  libskgxp10.so

FFFFFFFF7F500000          8          8          8          - rwx--    [ anon ]

FFFFFFFF7F522000         16         16          8          - rwx--  libskgxp10.so

FFFFFFFF7F600000        216        216          -          - r-x--  ld.so.1

FFFFFFFF7F700000          8          8          -          - r-x--  libsched.so.1

FFFFFFFF7F736000         16         16         16          - rwx--  ld.so.1

FFFFFFFF7F73A000          8          8          8          - rwx--  ld.so.1

FFFFFFFF7F7D0000          8          8          -          - rwxs-    [ anon ]

FFFFFFFF7FFC0000        256        256        256          - rw---    [ stack ]

---------------- ---------- ---------- ---------- ----------

        total Kb    2954192    2948592       7160    2818056

Scripts & Tools

1. NMUPM utility (Oracle Support)

This Solaris script is provided at Oracle Metalink article “How to Check the Host Memory Usage on Solaris via NMUPM Utility [ID 741004.1]”  and it can be used to obtain the average memory usage for Solaris machine for a pre-defined interval of time. The script uses the same procedure as that followed by the Oracle Enterprise Manager (OEM) agent.

#!/bin/ksh

PAGESZ="/usr/bin/pagesize"

BC="/bin/bc"

SCALE=2

WAIT=300

MAXCOUNT=3

NMUPM="$ORACLE_HOME/bin/nmupm osLoad"

echo "Calulates average memory (interval $WAIT (s)) usage on Solaris using nmupm"

PAGESIZE=`$PAGESZ`

result1=`$NMUPM | awk -F"|" '{print $14 }'`

REALMEM=`$NMUPM | awk -F"|" '{print $13 }'`

#echo $result1

X=0

while [ $X -le $MAXCOUNT ]

do

sleep $WAIT

result2=`$NMUPM | awk -F"|" '{print $14 }'`

#echo $result2

DIFF="($result2 - $result1) * $PAGESIZE / 1024 / $WAIT"

RESULT=$($BC << EOF

scale=$SCALE

(${DIFF})

EOF)

MEMREL="$RESULT / $REALMEM * 100"

MEMPCT=$($BC << EOF

scale=$SCALE

(${MEMREL})

EOF)

#echo $result1

echo "Memory $REALMEM [kB] Freemem $RESULT [kB] %Free $MEMPCT"

result1=$result2

X=$((X+1))

done

The typical output by running above script looks like below.

# ./nmupm_mem.sh

Calulates average memory (interval 10 (s)) usage on Solaris using nmupm

Memory 33325056.000000 [kB] Freemem 7497900.80 [kB] %Free 22.00

Memory 33325056.000000 [kB] Freemem 7497598.40 [kB] %Free 22.00

Memory 33325056.000000 [kB] Freemem 8245396.00 [kB] %Free 24.00

Memory 33325056.000000 [kB] Freemem 7491165.60 [kB] %Free 22.00

Please be noted that the value from the script and that in the OEM console may not match exactly as the script provides realtime values as the OEM console will shows the last value uploaded by the grid agent, but there should not be much difference between them. From running this script, we can briefly know the how much memory is utilized and how much are still free to use.


This entry was posted in My Reference, Oracle Point and tagged , , . Bookmark the permalink.

23 Responses to Different Ways to Check Memory Usage on Solaris Server

  1. We are a gaggle of volunteers and starting a brand new scheme in our community. Your website provided us with useful information to work on. You have performed an impressive activity and our entire group might be thankful to you.

  2. Deepak says:

    Content is good. but why you format everything in italic. Its makes reading little uncomfortable.

  3. Devesh vijay . says:

    how to check in g.b RAM size in Solaris …….

  4. Ajit Bhat says:

    Good content.
    Very helpful

  5. bigiah says:

    Thank you.
    The best article so far.
    Very helpful.

  6. Venkat says:

    A really good and brief article

  7. Anoop says:

    A very helpful article..

  8. SYED HADI AL HASSANI says:

    your contents are good and very helpful.
    Thanks

  9. Mario IBM says:

    Precious text. Show us quality like that. All the readers will benefit from the given post.

  10. Debojyoti says:

    Gr8, the only site where i found the command to find memory consumption by applications sorted in ascending order on solaris

  11. Ratan says:

    You provided quick reference to check memory consumption on the Unix system.
    This was very helpfull to me. Thank You.

  12. Mahua says:

    Very helpful.

  13. Precisely what I was searching for, appreciate it for posting.

  14. Shekhar says:

    Very useful content

  15. AhmadHassaan says:

    interesting…..thanks a lot

  16. John Carew says:

    Thank you very much for posting this blog.

    It’s been very useful as I start to review memory leaks and usage for applications running on solaris 5.9.

  17. Kirill says:

    prstat FTW! Thanks :)

  18. Surendra says:

    helped me to understand memory usage….

  19. GopiKrishna says:

    Good stuff…helped me a lot….Thank you…

  20. I got this web page from my pal who told me concerning this site and now this time I am visiting this
    web site and reading very informative articles
    at this place.

  21. I got this web page from my friend who told me about this
    web site and now this time I am visiting this web site and reading very informative articles or reviews at this place.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>