Three months after the Apache Foundation revealed the infamous Lo4j vulnerability [CVE-2021-44228] and released a fix for this, more than 4 out of 10 downloads of the logger from the Maven Central Java Package Repository continue to be known vulnerable versions.
A dashboard that Maven Central administrator Sonatype launched shortly after announcing the so-called Log4Shell flaw shows that 41% of Log4j packages downloaded between February 4 and March 10, 2022 are versions prior to Log4j 2.15.0. This is the patched version of the logging tool that the Apache Foundation released on December 10, 2021, when the Log4Shell flaw was first exposed. After that, the Foundation released two more updates to fix two later – and relatively less severe – vulnerabilities that were discovered in the logging tool just days after Log4Shell was disclosed.
Sonatype Dashboard showed that there have been over 31.4 million Log4j downloads in total since December 10, 2021. It is unknown how many of these versions are vulnerable, but based on the latest download statistics, the number could well be close to or above 10 million.
Log4j and layers
So why are organizations and developers downloading known vulnerable versions of Log4j packages, and why are these versions available for downloads in the first place, especially given the prevalence of the flaw and the relative ease with which which it can be exploited?
Travis Smith, VP of Malware Threat Research at Qualys, shares a few reasons why downloads continue. “The main culprit is probably the automated build systems, which are configured to download a specific version of their dependencies,” he says. Less maintained projects can automatically download a specific version to avoid conflicts with updated software. “If the maintainer of this software has not paid attention to the news surrounding Log4j, its application is left open to the risk of exploitation.”
The fact that Log4j is an integral part of many Java applications – and often buried in multiple layers – has made it extremely difficult for many organizations to detect and fix the issue. “One could infer that many of the current downloads of the vulnerable version are for projects that can’t justify the time it takes to upgrade,” Smith says.
Another explanation for the high percentage of vulnerable Log4j downloads, according to Smith, could be that researchers do it to test defenses, and adversaries do it to test their exploits.
Ilkka Turunen, Technical Field Manager at Sonatype, explains that another problem is the lack of fundamental software supply chain management in many organizations. Without adequate software composition analysis tools and software BOMs, organizations can struggle to determine which components have been released in which releases.
“The difficult part we observed with many organizations about the Log4j fire drill was a complete lack of knowledge and visibility of the third-party components used in their production,” says Turunen. Organizations often do not understand the content of their applications and therefore are unable to quickly target affected applications and perform upgrades. “In short, many companies are still trying to build their software inventory before they start to respond.”
Recent data that Qualys pulled from its cloud security platform actually suggests that around 30% of Log4j instances on the internet remain vulnerable to exploits, according to the company. “Apache Foundation’s Log4J is used in myriad places and comes with hundreds of packages,” says Mike Parkin, senior technical engineer at Vulcan Cyber. “That wide usage was part of what made it such a significant vulnerability in the first place.”
The fact that not all developers implement Log4J in their packages in the same way is another reason why patching has become a major problem, he says.
Why are vulnerable versions of Log4J still available?
Meanwhile, the reason vulnerable Log4j packages still remain available for download through Maven Central is due to software dependencies, Smith and others said. Many software depend on vulnerable versions of Log4j, and suddenly removing them could cause systems to crash. An analysis by Google researchers a week after the flaw was revealed Log4Shell showed that some 17,000 Java packages on Maven Central contained the vulnerability. At that time, Google discovered that patched versions were available for 25% of affected packages. Since then, it is likely that many more have been fixed.
Still, removing vulnerable versions from the Maven repository is risky. “When we became the administrators of Maven Central, one of the key commandments we put in place was ‘you shall not break a build,'” says Turunen. “While it may be tempting to assert our own judgment, which could eliminate vulnerabilities, what is actually best for the community is for everyone to make their own judgment.”
Brian Fox, co-founder and CTO of Sonatype, says Maven Central is more like a gas pipeline than a gas station where a user can choose their octane level every time. “If we removed those uploads from the repository, every build in the world that looked for it would suddenly fail,” he says.