Malware obfuscation, encoding and encryption (2024)

Malware analysis

Malware is complex and meant to confuse. Many computer users think malware is just another word for “virus” when a virus is actually a type of malware. And in addition to viruses, malware includes all sorts of malicious and unwanted code, including spyware, adware, Trojans and worms. Malware has been known to shut down power grids, steal identities and hold government secrets for ransom.

The swift detection and extraction of malware is always called for, but malware isn’t going to make it easy. Malware is mischievous and slippery, using tricks like obfuscation, encoding and encryption to evade detection.

Malware obfuscation

Understanding obfuscation is easier than pronouncing it. Malware obfuscation makes data unreadable. Nearly every piece of malware uses it.

The incomprehensible data usually contains important words, called “strings.” Some strings hold identifiers like the malware programmer's name or the URL from which the destructive code is pulled. Most malware has obfuscated strings that hide the instructions that tell the infected machine what to do and when to do it.

Obfuscation conceals the malware data so well that static code analyzers simply pass by. Only when the malware is executed is the true code revealed.

Simple malware obfuscation techniques

Simple malware obfuscation techniques like exclusive OR (XOR), Base64, ROT13 and codepacking are commonly used. These techniques are easy to implement and even easier to overlook. Obfuscation can be as simple as interposed text or extra padding within a string. Even trained eyes often miss obfuscated code.

The malware mimics everyday use cases until it is executed. Upon execution, the malicious code is revealed, spreading rapidly through the system.

Advanced malware obfuscation techniques

Next-level malware obfuscation is active and evasive. Advanced malware techniques, like environmental awareness, confusing automated tools, timing-based evasion, and obfuscating internal data, allow malware to lurk within operating environments and fly under the radar of reputable antivirus software.

Some malware thrives on bait-clicking users into downloading malware files or opening malicious webpages while others intercept traffic and inject malware, achieving vast and swift impact.

Using simple ciphers and encoding schemes for malware obfuscation

The name of the game is evasion. Malware puzzles with simple ciphers and encoding schemes. Making malware difficult to detect and decode delays analysis and remedy, as malicious code continually works behind the scenes.

Malware encoding

Base64 is often used to encode malware. It sounds like a covert military language but it’s actually a common binary-to-text encoding scheme. It was built to encode and decode in the service of the Multipurpose Internet Mail Extensions (MIME) internet standard which formats emails for transmittal.

The essential purpose of Base64 is to standardize code. Malicious programmers take advantage of this by injecting false character strings that mimic standard Base64. The code looks standard and functions normally until it’s decoded

Decoding malware

An offensive strategy is also needed to combat malware encoding, encryption and obfuscation. Software used to decode malware is meant to reveal malware in logical and straightforward ways prior to malware execution.

After execution, malware spreads and code becomes unreadable. Problematic text strings are first isolated, then search-and-replace techniques are used to eliminate malicious code. Once removed, you can see a clearer picture of the clean code.

Malware encryption

In the traditional sense, malware encryption is the process of encoding information so only authorized parties can access the data in a readable format.

When malware encryption is used for malicious intent, it is called ransomware. Ransomware holds files hostage using encryption. When the ransom payment is received, the files are decrypted and the user regains access. Today’s malware creators typically ask for payment in the form of cryptocurrency or credit card.

Malware most often infects systems when phishing schemes or other email tactics, posing as legitimate email, convince a user to click on a link or download a file.

Reverse engineering and common crypto algorithms

Reverse engineers, working backward from infection, use disassembly tools to reveal the file modifications that occur when malware attacks. Encryption is often used by malicious programmers to prevent disassembly and thereby prevent reverse engineering.

It’s helpful if reverse engineers understand the common crypto algorithms used to create symmetrically encrypted code. If the malware encryption was created using a common crypto algorithm, the code can be decrypted and reverse engineering can continue. If the cryptography is asymmetrical, however, the algorithm won’t offer the key to decryption.

Conclusion

Malware is sly and diverse in purpose but malicious code must work within established coding constructs. Veering outside the format of standard strings leaves malware open to discovery. Malicious programmers avoid this by using obfuscation, encoding and encryption to mimic and blend with surrounding code or by making code so unreadable that it can’t be deciphered.

Sometimes, ducking discovery only takes simple obfuscation like codepacking or encoding with Base64. The growth of malware analysts’ knowledge and experience and their tools growth in sophistication requires malware to adapt. Malware must employ advanced obfuscation techniques, cryptography and complex ciphers to avoid detection.

Sources

Malware Analysis, Part 1: Understanding Code Obfuscation Techniques, Vade Secure

Malware obfuscation techniques: four simple examples, andreafortuna.org

The Four Most Common Evasive Techniques Used by Malware, Tripwire

Decoding Complex Malware – Step-by-Step, Sucuri Blog

Encryption 101: a malware analyst’s primer, Malwarebytes Labs