Paging Space Tips

About this document
Related documentation
Paging space considerations
Options for calculating paging space requirements
Things to consider when creating or enlarging paging space
Determining if more paging space is needed
Adding or increasing a paging space
Tuning paging space thresholds

About this document

This document contains tips for allocating paging space on the system. The information contained in this document is valid for 4.3.X.

The following can be found at:
http://9.53.35.177:80/techlib/manuals/adoclib/aixbman/baseadmn/toc.htm AIX Version 4.3 System Management Guide: Operating System and Devices Chapter 8. Paging Space and Virtual Memory

The amount of paging space required depends on the types of activities performed on the system. If paging space runs low, processes may be lost, and if paging space runs out, the system may panic. When a paging-space low condition is detected, additional paging space should be activated.

The system monitors the number of free paging space blocks and detects when a paging space shortage exists. When the number of free paging space blocks falls below a threshold known as the paging space warning level, the system sends the SIGDANGER signal to all processes except the kprocs process. If the shortage continues, free paging space blocks can fall below a second threshold known as the paging space kill level. In this event, the SIGKILL signal is sent to processes that are major users of paging space and do not have a signal handler for the SIGDANGER signal. (The default action for the SIGDANGER signal is to ignore the signal.) The system continues sending SIGKILL signals until the number of free paging space blocks is above the paging space kill level.

You can ensure the existence of sufficient paging space for processes that dynamically allocate memory by monitoring the paging space levels with the psdanger subroutine or by using special allocation routines. The disclaim subroutine can be used to prevent processes from ending when the paging space kill level is reached. To do this, define a signal handler for the SIGDANGER signal and release memory and paging space resources allocated in the processes data and stack areas and in shared memory segments.

For more information on persistent and working segments, request The AIX Virtual Memory Manager (VMM) document.

Options for calculating paging space requirements

Paging space requirements are unique for each system, depending on the applications that are running, the number of active users, and other factors. With the appearance of large amounts of RAM and database applications, previous paging space rules of thumb have become invalid.

Option 1

Systems with large amounts of memory typically do not need large amounts of paging space. In a persistent storage environment, where the system hosts a few small programs and a large amount of data, the system may need less than one times (1X) its RAM size for paging space. For example, a 100GB database server that runs on a system with 16GB of RAM and uses only 2GB of working storage does not need 16GB, or even 8GB, of paging space. Because the 100GB database is mostly persistent storage and requires little or no paging space, it needs only the amount of paging space that allows all the working storage to be paged out to disk.

Option 2

The 1X RAM rule is suggested for use with less than or equal to 4GB of RAM. However, the paging space will have to be monitored during a period of heavy load to establish whether the paging space size is sufficient. Check the npswarn value of the vmtune command output and compare this value to the %used value of the lsps command output. When the %used value is equivalent to the npswarn value, then SIGDANGER signals are sent to processes. At this point, it is a good idea to increase or add another paging space. The npswarn value is explained in the Tuning paging space thresholds section of this document.

Option 3

For RAM sizes greater than 4GB, such as 16GB, 32GB or even 96GB, memory requirements for applications would have to be researched in order to approximate the recommended paging space sizes. When researching these memory requirements , keep in mind how the paging space will be allocated, that is, deferred or late.

Things to consider when creating or enlarging paging space

Before creating a new paging space or enlarging an existing paging space, consider the following:

If a disk drive containing an active hd6 paging space logical volume is removed from the system, the system will crash.
Do not put more than one paging space logical volume on a physical volume.When you have paging spaces on multiple physical volumes, if you add more than one paging space to one of the physical volumes, the paging activity is no longer spread equally across the physical volumes.
All processes started during the boot process are allocated paging space on the default paging space logical volume (hd6). When additional paging space logical volumes are activated, paging space is allocated in a "round robin" manner, in 4KB chunks.
Avoid putting a paging space logical volume on the same physical volume as a heavily active logical volume, such as that used by a database.
It is not necessary to put a paging space logical volume on each physical volume.
Make each paging space logical volume roughly equal in size.
If paging spaces are of different sizes, and the smaller ones become full, paging activity will no longer be spread across all of the physical volumes.
Do not extend a paging space logical volume onto multiple physical volumes.
For best system performance, put paging space logical volumes on physical volumes that are each attached to a different disk controller.
Do not add paging space logical volumes to ESS, EMC or RAID arrays.

NOTE: If system is paging enough to cause an I/O bottleneck, tuning the location of the paging space is not the answer.

In this case, please consult Chapter 7, "Monitoring and Tuning Memory Use of Performance Management Guide", in the Performance Management Guide at this location:

http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/aixbman/prftungd/toc.htm

Determining if more paging space is needed

Allocating more paging space than necessary results in unused paging space that wastes disk space. However, allocating too little paging space can result in one or more of the avoidable symptoms listed below. Use the following guidelines for determining the necessary paging space:

Enlarge paging space if any of the following messages are displayed on the console or in response to a command on any terminal:

INIT: Paging space is low

ksh: cannot fork no swap space

Not enough memory

Fork function failed

fork () system call failed

Unable to fork, too many processes

Fork failure - not enough memory available

Fork function not allowed. Not enough memory available.

Cannot fork: Not enough space

SIGKILL

Add a paging space if the average of the %Used column in the output of the lsps -a command is greater than 80.
Add a paging space if the %Used column in the output of the lsps -s command is greater than 80.
NOTE: Only extend a paging space as a last option.

Use the following commands to determine if you need to make changes regarding paging space logical volumes:

iostat Check the tm_act field for the hdisk containing the paging space for a high percentage relative to the other hdisks

vmstat Assure fr/sr columns of the vmstat page field do not consistently exceed the ratio of 1:4.

lsps Use the -a flag to list all characteristics of all paging spaces. The size is given in megabytes. Use the -s flag to list the summary characteristics of all paging spaces. This information consists of the total paging space in megabytes and the percentage of paging space currently assigned (used). If the -s flag is specified, all other flags are ignored.

Adding or increasing a paging space

Follow these steps to add a paging space. Please note that the command output shown in this example may differ from the command output on your system.

Check existing paging spaces and available physical volumes.

The following command lists characteristics for all existing paging spaces:

#lsps -a
Page Space  Physical Volume   Volume Group    Size    %Used  Active  Auto  Type
Paging02      hdisk2             rootvg       512MB     3     yes     yes   lv
Paging01      hdisk3             testcase     512MB     3     yes     yes   lv
Paging00      hdisk1             rootvg       512MB     2     yes     yes   lv
Hd6           hdisk0             rootvg       512MB     3     yes     yes   lv

The following command lists the available physical volumes:

#lspv
hdisk0         000336524e264c40    rootvg
hdisk1         00033652f9fe5c81    doomvg
hdisk2         00302593eb30798f    none
hdisk3         00033652fa08edca    testcase

Make sure the physical volume where the paging space will be assigned is part of a volume group (check column 3). In this example, paging space can be created on all hdisk?s, except hdisk2.

The following command displays detailed information about the physical volume within a volume group where you plan to assign the paging space.

#lspv hdisk3
PHYSICAL VOLUME:    hdisk3                   VOLUME GROUP:     testcase
PV IDENTIFIER:      00033652fa08edca         VG IDENTIFIER     0008508436f7d210
PV STATE:           active
STALE PARTITIONS:   0                        ALLOCATABLE:      yes
PP SIZE:            8 megabyte(s)            LOGICAL VOLUMES:  3
TOTAL PPs:          268 (2144 megabytes)     VG DESCRIPTORS:   2
FREE PPs:           201 (1608 megabytes)
USED PPs:           67 (536 megabytes)
FREE DISTRIBUTION:  00..00..00..00..07
USED DISTRIBUTION:  54..54..53..53..47

Make a note of the following items
PP SIZE: (8 megabyte(s) in this example)
FREE PPs: (201 (1608 megabytes) in this example)
Decide the size of the new paging space, remembering that keeping paging space sizes equal improves system performance. This example creates a paging space of 512 MB. Use smittyfastpath to display the Volume Group name window.
Do one of the following:
#smitty mkps
OR
```
#smitty
  >>System Storage Management
  >>Logical Volume Manager
  >>Paging Space
  >>Add another paging space.
```
Choose the volume group name of which the physical volume is a part. The next screen is displayed, with the SIZE of paging space field highlighted. This value entered in this field is the number of logical partitions.
To calculate the # of logical partitions, divide the number of MBs to be created by the PP SIZE value. This value equals the number of logical partitions. Enter this number in the highlighted field.
```
# of logical = Prospective Paging Space Size in MBs
Partitions             PP SIZE
```
Select the physical volume where you want to assign the new paging space. The F4 key can be used to assist in selecting available physical volumes.
Using the Tab key, select yes for the option to start using this paging space now and yes for the option to use this paging space each time the system is restarted. Press the Enter key after selecting these two options.
Run the lsps -a command to compare the new paging space with any others on the system.
- Make sure the value in both the Active and Auto columns is set to yes.
- Verify that the value in the %Used column is at least 1%. The system uses the paging space in a round robin fashion and %Used value increases over time.

No reboot is required, and can you can complete this procedure on a production system.

Increase paging space

Follow these steps to increase the size of an existing paging space:

Determine the PP SIZE and FREE PP values for the disk where you want to increase the paging space. Follow the instructions in Steps 1 through 3 of the previous procedure.
Decide the number of megabytes by which you want to increase the paging space.
Divide the number of megabytes by the PP SIZE value to determine the equivalent number of logical partitions needed to increase the paging space.
Use the following smitty fastpath command to display the dialog box for making changes to an existing paging space:
```
smitty chps
```
Complete the entries on the dialog box as follows:
- Enter the number of logical partitions needed to increase the paging space (as determined in Step 3).
- Set the value to yes in the Use this paging space each time the system is RESTARTED? field.
Press Enter to execute the dialog.
Run the lsps -a command to verify the size of the paging space and to check that the value in both the Active and Auto columns is set to yes.

Tuning paging space thresholds

For additional information, please Chapter 7, "Monitoring and Tuning Memory Use", in the Performance Management Guide at this location:

http://www.rs6000.ibm.com/doc_link/en_US/a_doc_lib/aixbman/prftungd/toc.htm

If available paging space becomes depleted, the operating system attempts to release resources as follows:

First, by warning processes to release paging space
Then, if there is still insufficient paging space for current process, by killing processes.

The VMM uses the values of two parameters that specify the thresholds for sending warning or kill signals to processes:

npswarn The value of this parameter specifies the paging space warning threshold, below which warning signals are sent to processes.

npskill The value of this parameter specifies the paging space kill threshold, below which kill signals are sent to certain processes.

Choosing npswarn and npskill settings

These values for the npswarn and npskill parameters are set by means of arguments to the vmtune command:

Parameter	vmtune flag	Description
npswarn	-w	Specifies the number of free paging space pages at which the operating system begins sending the SIGDANGER signal to processes. If the npswarn threshold is reached and a process is handling this signal, the process can choose to ignore the signal or do some other action, such as exit or release memory by using the disclaim() subroutine. The default value in operating system version 4 is determined by the following formula: npswarn = 4*npskill The value of npswarn must be greater than zero and less than the total number of paging space pages on the system. This parameter can be changed by using the vmtune -w command.
npskill	-k	Specifies the number of free paging space pages at which the operating system begins killing processes. If the npskill threshold is reached, a SIGKILL signal is sent to the youngest process. Processes that are handling SIGDANGER or processes that are using the early page-space allocation (paging space is allocated as soon as memory is requested) are exempt from being killed. The formula to determine the default value of npskill is as follows: npskill = number_of_paging_space_pages/128 The npskill value must be greater than zero and less than the total number of paging space pages on the system. This parameter can be changed by using the vmtune -k command.

Parameter

vmtune flag

Description

npswarn

-w

Specifies the number of free paging space pages at which the operating system begins sending the SIGDANGER signal to processes. If the npswarn threshold is reached and a process is handling this signal, the process can choose to ignore the signal or do some other action, such as exit or release memory by using the disclaim() subroutine. The default value in operating system version 4 is determined by the following formula:

npswarn =  4*npskill

The value of npswarn must be greater than zero and less than the total number of paging space pages on the system. This parameter can be changed by using the vmtune -w command.

npskill

-k

Specifies the number of free paging space pages at which the operating system begins killing processes. If the npskill threshold is reached, a SIGKILL signal is sent to the youngest process. Processes that are handling SIGDANGER or processes that are using the early page-space allocation (paging space is allocated as soon as memory is requested) are exempt from being killed. The formula to determine the default value of npskill is as follows:

npskill =  number_of_paging_space_pages/128

The npskill value must be greater than zero and less than the total number of paging space pages on the system. This parameter can be changed by using the vmtune -k command.

Example

Real memory = 16GB
Paging space = 4096MB
Convert paging space to 4KB pages.
4096MB *(1024KB/4KB)=1048576 4KB pages
npskill = number_of_paging_space_pages/128
            = 1048576 pages/128
            = 8192 pages
npswarn = 4*npskill
              = 4*8192
              = 32768 pages
Npskill % of paging space = ((1048576 - 8192)/1048576)*100
                                           = 99.2
Npswarn % of paging space = ((1048576 - 32768)/1048576)*100
                                              = 96.8
Example of a 8GB paging space:
Npskill % of paging space = ((2097152 - 16384)/2097152)*100
                                           = 99.2
Npswarn % of paging space = ((2097152 - 65536)/2097152)*100
                                  = 96.8
Example of a 16GB paging space:
Npskill % of paging space = ((4194304 - 32768)/4194304)*100
                               = 99.2
Npswarn % of paging space =((4194304 - 131072)/4194304)*100
                                  = 96.8

The npswarn and npskill default value percentages for paging spaces of 4GB, 8GB and 16GB all are equivalent. Notice the npswarn default is 96.8%. This percentage translates to the point at which the paging space usage percentage will send SIGDANGER signals to marked processes. Then when the paging space usage percentage reaches 99.2, the SIGKILL signal will sent to the youngest process. To gain more notice time, decrease npswarn percentages to give a more advanced warning.

Other vmtune parameters

The following parameters are also set by arguments to the vmtune command:

Parameter	vmtune flag	Description
nokillroot	-n	By setting the nokillroot option to 1 with the command vmtune -n 1, processes owned by root will be exempt from being killed when the npskill threshold is reached. This option is only available in operating system version 4.3.3 and 4.3.3.1.
nokilluid	-n	By setting the nokilluid option to a nonzero value with the command vmtune -n, user IDs lower than this value will be exempt from being killed because of low page-space conditions. This option is only available in operating system version 4.3.3.2 and later.

For more information on the vmtune flags see Appendix G of the Performance Management Guide.

Tuning the fork() retry interval parameter with schedtune

If a process cannot be forked due to a lack of paging-space pages, the scheduler retries the fork five times. After each try, the scheduler delays for a default of 10 clock ticks.

The -f flag of the schedtune command specifies the number of clock ticks to wait before retrying a failed fork() call. For example, if a fork() subroutine call fails because there is not enough space available to create a new process, the system retries the call after waiting the specified number of clock ticks. The default value is 10, and because there is one clock tick every 10 ms, the system retries the fork() call every 100 ms.

If the paging space is only low due to brief, sporadic workload peaks, increasing the retry interval might allow processes to delay long enough to be released. See the following example:

     # /usr/samples/kernel/schedtune -f 15

In this way, when the system retries the fork() call, there is a higher chance of success because some processes might have finished their execution and, consequently, released pages from paging space.

[ Doc Ref: 90605219714826 Publish Date: Dec. 03, 2001]

iostat	Check the tm_act field for the hdisk containing the paging space for a high percentage relative to the other hdisks
vmstat	Assure fr/sr columns of the vmstat page field do not consistently exceed the ratio of 1:4.
lsps	Use the -a flag to list all characteristics of all paging spaces. The size is given in megabytes. Use the -s flag to list the summary characteristics of all paging spaces. This information consists of the total paging space in megabytes and the percentage of paging space currently assigned (used). If the -s flag is specified, all other flags are ignored.