Log4j: The vigilant Chinese, the brave Apaches, and our Christmas

November 24, 2021: Software developer Chen Zhaojun of Chinese cloud giant Alibaba discovers the cyber vulnerability of 2021: ocurring in the Apache Foundation's highly popular open source program Log4j, globally distributed. Our hero Chen reports it, as any responsible code developer would, to Apache, which sits in the USA. A few days later, he already registers the first active exploits by (Chinese!) hackers and notifies Apache again to hurry up with the patch. He is indeed a hero; the Chinese Cyber Security Law CCSL prohibits what he did, the uncoordinated reporting of zero-day vulnerabilities to anyone, especially abroad. His employer, China‘s Alibaba will, a month later, as a consequence, suffer a publicly ordered boycott by national government organizations. Does anybody here feel reminded of Alibaba, and the disappearance of founder Jack Ma for months a year ago?

Apache releases the first patch for the calamity shortly after, on 09.12.2021. As well as some emergency workarounds. However, both solutions are incomplete. They provide no security.

Warnings of active exploitation now pile up globally in the following days; the German Federal Office for Information Security BSI issues a 'yellow' warning on Friday, 10.12.2021, and upgrades to 'red' on Sunday. A rare occurrence.

In the German Federal Association of IT Users, VOICE e.V., we warned on Monday morning, via flash report. An emergency video conference of the association's member companies convenes two days later. Late in the evening, with 175 participants, mostly CIOs and CISOs. Burning platform!

During the next days, something unfolds, what is probably the biggest wave since a long time, but certainly for 2021. Almost daily, 'patches from Apache' follow, which claim to close the gaps that still exist. And the malicious exploitation is progressing in the meantime. On Monday, 20.12.2021, the first working day of our Christmas week, to top it all, a supposedly worm-capable, self-propagating malicious code appears on the worldwide web.  In the meantime, a major firewall provider is reporting more than 100,000 attack attempts per second on the devices it monitors. And so on.

The patch wave (with version 2.17) comes to a temporary halt on 27.12.2021, the day after our Christian Christmas.

Thousands of IT security managers and specialists around the world continue to feverishly investigate in which even remote code components of their complex corporate IT infrastructures Log4j could be hidden, in a race to patch this location as well.

Here, the opportunities and risks of open source use are reflected like in a magnifying glass. But, on the positive side, we will concede without envy: only one approximative month of response time between the first vulnerability report (24.11.) and the last patch (27.12.) is quick, and oftentimes not  being reached by many proprietary software products, e.g. of our large, often US-based, software vendors.

So this is how our Christmas month of December went. Let's look at the calamity with a little humor at the end: We can be grateful to the vigilant Chinese Chen for reporting. We can thank the brave Apaches for responding so quickly. But: our (Christian) Christmas was impaired, namely by the feverish search for needles in the haystack, by built-up emergency monitoring, by monitoring capacity, and our perceived insecurity.

What remains to be hoped for: that security will be restored by the end of January. Because: on February 1, 2022, begins – the (Chinese) Christmas!  We wish Mr. Chen one.

Author: Dirk Michael Ockel