vCenter Server Applicance disk space is full

APPLIES TO

VMware vCenter Server Applicance - 7.0.2 - 17920168

SYMPTOMS

The /storage/log is full.

root@localhost [ ~ ]# df -h
Filesystem                                Size  Used Avail Use% Mounted on
devtmpfs                                  9.3G     0  9.3G   0% /dev
tmpfs                                     9.4G   52K  9.4G   1% /dev/shm
tmpfs                                     9.4G  1.2M  9.4G   1% /run
tmpfs                                     9.4G     0  9.4G   0% /sys/fs/cgroup
/dev/mapper/vg_root_0-lv_root_0            47G   10G   35G  23% /
tmpfs                                     9.4G  288K  9.4G   1% /tmp
/dev/mapper/core_vg-core                   49G  969M   46G   3% /storage/core
/dev/mapper/updatemgr_vg-updatemgr         98G  2.7G   91G   3% /storage/updatemgr
/dev/mapper/netdump_vg-netdump            985M  2.5M  915M   1% /storage/netdump
/dev/mapper/log_vg-log                    9.8G  9.7G   10M 100% /storage/log
/dev/sda3                                 488M   40M  414M   9% /boot
/dev/mapper/autodeploy_vg-autodeploy      9.8G   37M  9.3G   1% /storage/autodeploy
/dev/mapper/imagebuilder_vg-imagebuilder  9.8G   37M  9.3G   1% /storage/imagebuilder
/dev/mapper/lifecycle_vg-lifecycle         98G  3.1G   90G   4% /storage/lifecycle
/dev/mapper/db_vg-db                      9.8G  2.8G  6.6G  30% /storage/db
/dev/mapper/dblog_vg-dblog                 15G   89M   14G   1% /storage/dblog
/dev/mapper/archive_vg-archive             49G   43G  3.7G  93% /storage/archive
/dev/mapper/vtsdblog_vg-vtsdblog           15G   57M   14G   1% /storage/vtsdblog
/dev/mapper/vtsdb_vg-vtsdb                 25G   76M   24G   1% /storage/vtsdb
/dev/mapper/seat_vg-seat                   25G  797M   23G   4% /storage/seat
/dev/sda2                                  10M  2.2M  7.9M  22% /boot/efi

CAUSE

vCenter has a large number of localhost_access log files generated under /storage/log/vmware/eam/web/.

SOLUTION

  1. Remove the log files. # rm -f /storage/log/vmware/eam/web/localhost_access.*.log
  2. Backup the file. # cp -a /usr/lib/vmware-eam/web/conf/server.xml{,.bak}
  3. Stop the ESX Agent Manager service: # service-control --stop vmware-eam
  4. Edit /usr/lib/vmware-eam/web/conf/server.xml.
  5. Locate the line <Valve className="org.apache.catalina.valves.AccessLogValve".
  6. Modify the numberOfDays : maxDays="<numberOfDays>" or add maxDays="" if the parameter is not present within the brackets.
  7. Save the change changes, and restart the ESX Agent Manager service: # service-control --start vmware-eam

REFERENCE

・ Failed to start file system check on dev_disk... error on Photon OS based virtual appliances (2149838).pdf |
・ Troubleshooting vCenter Appliance _storage_log directory is 80% or more full (83070).pdf |
・ vCenter Server Appliance disk space is full (76563).pdf |
・ vCenter Server _storage_log filling up due to localhost_access.log and catalina.log in sso and lookupsvc log directories (85475).pdf |
・ 503 Service Unavailable error on the vSphere Web Client when logging in or accessing the vCenter Server.pdf |