cve

Fixing Security Vulnerabilities in Linux

Security vulnerabilities are some of the hardest bugs to discover yet they can have the largest impact. At Ksplice, we spend a lot of time looking at security vulnerabilities and seeing how they are fixed. We use automated tools such as the Trinity syscall fuzzer and the Kernel Address Sanitizer (KASan) to aid our process. In this blog post we’ll go over some case studies of recent vulnerabilities and show you how you can avoid them in your code.

CVE-2013-7339 and CVE-2014-2678

These two are very similar NULL pointer dereferences when trying to bind an RDS socket without having an RDS device. This is an oversight that happens quite often in hardware-specific code in the kernel. It is easy for developers to assume that a piece of hardware always exists since all their dev machines have it, but that sometimes leads to other possible hardware configurations left untested. In this example the code makes a seemingly reasonable assumption that using RDS sockets without RDS hardware doesn’t really make sense.

The issue is pretty simple as we can see from this fix:

diff --git a/net/rds/ib.c b/net/rds/ib.c
index b4c8b00..ba2dffe 100644
--- a/net/rds/ib.c
+++ b/net/rds/ib.c
@@ -338,7 +338,8 @@ static int rds_ib_laddr_check(__be32 addr)
   ret = rdma_bind_addr(cm_id, (struct sockaddr *)&sin);
   /* due to this, we will claim to support iWARP devices unless we
      check node_type. */
-     if (ret || cm_id->device->node_type != RDMA_NODE_IB_CA)
+     if (ret || !cm_id->device ||
+         cm_id->device->node_type != RDMA_NODE_IB_CA)
                                   ret = -EADDRNOTAVAIL;

                                   rdsdebug("addr %pI4 ret %d node type %d\n",

Generally we are allowed to bind an address without a physical device so we can reach this code without any RDS hardware. Sadly, this code wrongly assumes that a devices exists at this point and that cm_id->device is not NULL leading to a NULL pointer dereference.

These type of issues are usually caught early in -next as that exposes the code to various users and hardware configurations, but this one managed to slip through somehow.

There are many variations of the scenario where the hardware specific and other kernel code doesn’t handle
cases which “don’t make sense”. Another recent example is dlmfs. The kernel would panic when trying to create a directory on it – something that doesn’t happen in regular usage of dlmfs.

CVE-2014-3940

This one is interesting and very difficult to stumble upon by accident. It’s a race condition that is only possible during the migration of huge pages between NUMA nodes, so the window of opportunity is *very* small. It can be triggered by trying to dump the NUMA maps of a process while its memory is being moved around. What happens is that the code trying to dump memory makes invalid memory accesses because it does not check the presence of the memory beforehand.

When we dump NUMA maps we need to walk memory entries using walk_page_range():


        /*
         * Handle hugetlb vma individually because pagetable
         * walk for the hugetlb page is dependent on the
         * architecture and we can't handled it in the same
         * manner as non-huge pages.
         */
        if (walk->hugetlb_entry && (vma->vm_start <= addr) &&
            is_vm_hugetlb_page(vma)) {
                if (vma->vm_end < next)
                        next = vma->vm_end;
                /*
                 * Hugepage is very tightly coupled with vma,
                 * so walk through hugetlb entries within a
                 * given vma.
                 */
                err = walk_hugetlb_range(vma, addr, next, walk);
                if (err)
                        break;
                pgd = pgd_offset(walk->mm, next);
                continue;
        }

When walk_page_range() detects a hugepage it calls walk_hugetlb_range(), which calls the proc’s callback (provided by walk->hugetlb_entry()) for each page individually:


        pte_t *pte;
        int err = 0;

        do {
                next = hugetlb_entry_end(h, addr, end);
                pte = huge_pte_offset(walk->mm, addr & hmask);
                if (pte && walk->hugetlb_entry)
                        err = walk->hugetlb_entry(pte, hmask, addr, next, walk);
                if (err)
                        return err;
        } while (addr = next, addr != end);

Note that the callback is executed for each pte; even for those that are not present in memory (pte_present(*pte) would return false in that case). This is done by the walker code because it was assumed that callback functions might want to handle that scenario for some reason. In the past there was no way for a huge pte to be absent, but that changed when the hugepage migration was introduced. During page migration unmap_and_move_huge_page() removes huge ptes:


if (page_mapped(hpage)) {

try_to_unmap(hpage,
TTU_MIGRATION|TTU_IGNORE_MLOCK|TTU_IGNORE_ACCESS);

page_was_mapped = 1;

}

Unfortunately, some callbacks were not changed to deal with this new possibility. A notable example is gather_pte_stats(), which tries to lock a non-existent pte:


        orig_pte = pte = pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl);

This can cause a panic if it happens during a tiny window inside unmap_and_move_huge_page().

Dumping NUMA maps doesn’t happen too often and is mainly used for testing/debugging, so this bug has lived there for quite a while and was made visible only recently when hugepage migration was added.

It’s also common that adding userspace interfaces to trigger kernel code which doesn’t get called often exposes many issues. This happened recently when the firmware loading code was exposed to userspace.

CVE-2014-4171

This one also falls into the category of “doesn’t make sense” because it involves repeated page faulting of memory that we marked as unwanted. When this happens shmem tries to remove a block of memory, but since it’s getting faulted over and over again shmem will hang waiting until it’s available for removal. Meanwhile other filesystem operations will be blocked, which is bad because that memory may never become available for removal.

When we’re faulting a shmem page in, shmem_fault() would grab a reference to the page:


static int shmem_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
{
[...]
error = shmem_getpage(inode, vmf->pgoff, &vmf->page, SGP_CACHE, &ret);

But because shmem_fallocate() holds i_mutex this means that shmem_fallocate() can wait forever until it can free up that page. This, in turn means that that filesystem is stuck waiting for shmem_fallocate() to complete.

Beyond that, punching holes in files and marking memory as unwanted are not common operations; especially not on a shmem filesystem. This means that those code paths are very untested.

CVE-2014-4943

This is a privilege escalation which was found using KASan. We’ve noticed that as a result of a call to a PPPOL2TP ioctl an uninitialized address inside a struct was being read. Further investigation showed that this is the result of a confusion about the type of the struct that was being accessed.

When we call setsockopt from userspace on a PPPOL2TP socket in userspace we’ll end up in pppol2tp_setsockopt() which will look at the level parameter to see if the sockopt operation is intended for PPPOL2TP or the underlying UDP socket:


   if (level != SOL_PPPOL2TP)
      return udp_prot.setsockopt(sk, level, optname, optval, optlen);

PPPOL2TP tries to be helpful here and allows userspace to set UDP sockopts rather than just PPPOL2TP ones. The problem here is that UDP‘s setsockopt expects a udp_sock:


 int udp_lib_setsockopt(struct sock *sk, int level, int optname,
                        char __user *optval, unsigned int optlen,
                        int (*push_pending_frames)(struct sock *))
 {
         struct udp_sock *up = udp_sk(sk);

But instead it’s getting just a sock struct.

It’s possible to leverage this struct confusion to achieve privilege escalation. We can overwrite the function pointer in the struct to point to code of our choice. Then we can trigger the execution of this code by making another socket operation. The piece of code that allowed for this vulnerability was added for convenience, but no one ever needed it, and it was never tested.

Conclusion

We hope that this exposition of straightforward and more subtle kernel bugs will remind of the importance of looking at code from a new perspective and encourage the developer community to contribute to and create new tools and methodologies for detecting and preventing bugs in the kernel.

Information on the SSL connection vulnerability of MySQL and MariaDB

Last  week, a SSL connection security vulnerability was reported for MySQL and MariaDB. The vulnerability states that since MariaDB and MySQL do not enforce SSL when SSL support is enabled, it’s possible to launch Man In The Middle attacks (MITM). MITM attacks can capture the secure connection and turn it into an insecure one, revealing data going back and forth to the server.
Issue resolution in MariaDB is visible through the corresponding ticket in MariaDB’s tracking system (JIRA): https://mariadb.atlassian.net/browse/MDEV-7937
The vulnerability affects the client library of the database server in both MariaDB and MySQL. But, the vulnerability does not affect all the libraries, drivers or connectors for establishing SSL connections with the server.
The vulnerability exists when the connection to the server is done through the client library libmysqlclient. This client library is provided with the database server and is a fork of the corresponding client library in MySQL. The client library is used by probably the most used tool, the MySQL Command-Line tool of which a forked version is shipped with MariaDB.
In addition to libmysqlclient, the MariaDB project provides the following connectors:

MariaDB Connector/C, https://mariadb.com/kb/en/mariadb/mariadb-connector-c/
MariaDB Connector/J (Java Client), https://mariadb.com/kb/en/mariadb/about-the-mariadb-java-client/
MariaDB Connector/ODBC, https://mariadb.com/kb/en/mariadb/mariadb-connector-odbc/

These connectors also support SSL connections to the database server and make use of the similar parameters etc. to establish secure connections. Here is an update on whether the connectors are affected or not:

Affected – MariaDB Connector/C is vulnerable in the same way as libmysqlclient
Not affected – MariaDB Connector/J does the right thing and aborts any unsecure connections if SSL is in use
Not affected – MariaDB Connector/ODBC does not currently support SSL

For MySQL’s Connector/J it is worth mentioning that it has two properties, “useSSL” and “requireSSL”. If “requireSSL” is selected, then unsecure connections are aborted.
Many of the tools that are used to connect to MariaDB or MySQL make use of libmysqlclient. Thus, when using these tools over an untrusted network, it’s highly recommended that you restrict network access as much as possible with normal means, even if you’re using SSL to connect to MariaDB or MySQL. Some best practices that are easy to put in place for decreasing the risk of MITM attacks include:

MAC address filtering
IP filtering
Setting up a VPN connection between the client and the server

Disabling remote connections to the server and Setting up MySQL port forwarding over an SSH tunnel.

Finally, since we’re in the middle of fixing the vulnerability in MariaDB, we appreciate your input regarding which versions of MariaDB that should get the fix backported. For background, the SSL support in MySQL (up until 5.7) and MariaDB is not enforceable. This is the intended MySQL behavior, implemented back in 2000, and clearly documented in the MySQL reference manual as:
“For the server, this option specifies that the server permits but does not require SSL connections.
For a client program, this option permits but does not require the client to connect to the server using SSL. Therefore, this option is not sufficient in itself to cause an SSL connection to be used. For example, if you specify this option for a client program but the server has not been configured to permit SSL connections, an unencrypted connection is used.”
MariaDB 5.5 and 10.0 are stable versions and behave as documented – they permit SSL, but do not require it. To enforce SSL, when the appropriate options are given, will change the behavior and might break existing applications where a mix of SSL and non-SSL connections are used. In MariaDB 10.1 this is not a problem since MariaDB 10.1 is still in beta, although it is very close to release candidate status. There we will introduce the fix. As for MariaDB 5.5 and 10.0, we are collecting input to determine whether we should change the behavior of 5.5 and 10.0. Please visit our website for more details, and share your feedback at: http://info.mariadb.com/ssl-vulnerability-mysql-mariadb
The initial reports on the vulnerability can be found through these sources:

A good report with more information on the vulnerability is provided through oCERT: http://www.ocert.org/advisories/ocert-2015-003.html
The official CVE report: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-3152

Let’s Play Vulnerability Bingo!

Dear Fellow System Administrators,

I like excitement in my life. I go on roller coasters, I ride my bike without a helmet, I make risky financial decisions. I treat my servers no differently. When my Linux vendor releases security updates, I think: I could apply these patches, but why would I? If I did, I’d have to coordinate with my users to schedule a maintenance window for 2am on a Sunday and babysit those systems while they reboot, which is seriously annoying, hurts our availability, and interrupts my beauty sleep (and trust me, I need my beauty sleep). Plus, where’s the fun in having a fully-patched system? Without open vulnerabilities, how else would I have won a ton of money in my office’s Vulnerability Bingo games?

vulnerability bingo card

How can I get in on some Vulnerability Bingo action, you ask? Simple: get yourself some bingo cards, be sure not to patch your systems, and place chips on appropriate squares when your machines are compromised. Or, as a fun variant, place chips when your friends’ machines get compromised! For the less adventurous, place chips as relevant Common Vulnerabilities and Exposures are announced.

What’s really great is the diversity of vulnerabilities. In 2009 alone, Vulnerability Bingo featured:

physically proximate denial of service attacks (CVE-2009-1046).

local denial of service attacks (CVE-2009-0322, CVE-2009-0031, CVE-2009-0269, CVE-2009-1242, CVE-2009-2406, CVE-2009-2407, CVE-2009-2287, CVE-2009-2692, CVE-2009-2909, CVE-2009-2908, CVE-2009-3290, CVE-2009-3547, CVE-2009-3621, CVE-2009-3620) coming in at least 5 great flavors: faults, memory corruption, system crashes, hangs, and the kernel OOPS!

And the perennial favorite, remote denial of service attacks (CVE-2009-1439, CVE-2009-1633, CVE-2009-3613, CVE-2009-2903) including but not limited to system crashes, IOMMU space exhaustion, and memory consumption!

How about leaking potentially sensitive information from kernel memory (CVE-2009-0676, CVE-2009-3002, CVE-2009-3612, CVE-2009-3228) and remote access to potentially sensitive information from kernel memory (CVE-2009-1265)?

Perhaps I can interest you in some privilege escalation (CVE-2009-2406, CVE-2009-2407, CVE-2009-2692, CVE-2009-3547, CVE-2009-3620), or my personal favorites, arbitrary code execution (CVE-2009-2908) and unknown impact (CVE-2009-0065, CVE-2009-1633, CVE-2009-3638).

Sometimes you get a triple threat like CVE-2009-1895, which “makes it easier for local users to leverage the details of memory usage to (1) conduct NULL pointer dereference attacks, (2) bypass the mmap_min_addr protection mechanism, or (3) defeat address space layout randomization (ASLR)“. Three great tastes that taste great together — and a great multi-play Bingo opportunity!

Linux vendors release kernel security updates almost every month (take Red Hat for example), so generate some cards and get in on the action before you miss the next round of exciting CVEs!

Happy Hacking,

Ben Bitdiddle

System Administrator

HackMe Inc.

~jesstess

TEL/電話+86 13764045638
Email service@parnassusdata.com
QQ 47079569