Customize OVAL for malware identification or audit

Abstract 

In this post, I will talk about the basic structure of OVAL so you can create your own inventory, assessment, or incident response tool. You should also realize that basic programming skill is required to be successful in this industry. You should have a basic understanding of XML structure and editing skills. OVAL allows a user to create an automated testing framework for identifying malware files or registry entries. I will show you how to create a registry entry that will serve as a malicious entry that you as an incident responder identified in one system and would like to test the whole environment to see how far it spread or how many systems are infected. This way, you can respond to incidents instantly without waiting for solutions from others. I will also show you how to locate a “malicious” file that can be a malware on a system, but it will not do harm until someone executes that program. Thus, identifying it on a system before it is executed can be applied as a preventative measure even if your anti-virus software is missed to identify it. OVAL allows advanced matching patterns like Regular Expressions, but we’ll not going to cover that topic. OVAL can also be used to measure compliance and audit systems setup by the IT department or use it to certify a system before attaching it to the network.

 Concept 

Read the tutorial on http://oval.mitre.org/language/about/definition.html that will explain the basic concept of creating this simple example.

 

OVAL structure is based on rules you establish in your definitions file that is in a form of XML. In our example, we’ll have two definitions; one to identify a registry value and the other to identify a version of a file. Definitions have IDs in our case, they will be oval:zoltan:def:1 and oval:zoltan:def:2. Each definitions pointing to a criteria that needs to be tested for validity. Thus, the first definition oval:zoltan:def:1 will test oval:zoltan:tst:1 and the second definition oval:zoltan:def:2 will test oval:zoltan:tst:2. Each tests will perform its test by referring to an object and a state for that object. oval:zoltan:tst:1 will receive its object information from oval:zoltan:obj:1 and its state information for that object from oval:zoltan:ste:1. oval:zoltan:tst:2 will receive its object information from oval:zoltan:obj:2 and its state information for that object from oval:zoltan:ste:2.

Finally, we arrive at the last structures of this reference lists and we can identify the specific information the objects are set to and the states we are looking to identify.

Thus, object id oval:zoltan:obj:1 is creating a registry object based on the registry key HKEY_LOCAL_MACHINE\SOFTWARE\oval\example and the state oval:zoltan:ste:1 is looking to see if the key value is “Hello World” or not. In our case, if Hello World is not the value under this key, then we’ll receive a FALSE result from our report, thus we are not vulnerable to this “malware”, but if we create this value ( assume a malware created this value without our knowledge ) then the result will be TRUE meaning that you are vulnerable and you should do something to mitigate this problem.

Object id oval:zoltan:obj:2 first tries to locate the system directory that should be C:\WINDOWS in most systems by consulting a variable oval:zoltan:var:1 where oval:zoltan:var:1 needs to find the system drive by looking at in the registry defined in oval:zoltan:obj:3, HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\SystemRoot. The value returned should be C:\WINDOWS and the literal component adds the value system32 to it to come up with the final result of C:\WINDOWS\SYSTEM32 as a directory for the file we are looking for to verify. After we came up with the path, you just need to test if the filename NOTEPAD.EXE is in that path or not. If the file object is located, then you can check to see what is its version that is specified in file state oval:zoltan:ste:2. If the file state matches, then you will have a result of TRUE in your final report, if it does not match, then you will have a FASLE result. TRUE would mean that you have a version of the file on your system that is vulnerable so you can start your mitigation process. FALSE would indicate that you are not vulnerable to the specific threat. You do not have to find out the version exactly, but you can use a less than or a greater than variable to cover more than just one version of vulnerable products.

Procedure 

Download and install notepad++ from http://notepad-plus-plus.org/download . Notepad++ will give you a better structural representation of you xml file than other tools or simple tools like notepad. Do not use complex tools like MS Word for this exercise.

Copy and save the supplied definition at the end of this document into a file ( zoltan-reg-file.xml ) and open it in notepad++. You should see the contents of the file in a user friendly colored format like shown below. ( you might have to select ( Language-> XML first )  

      Figure 1: XML definition file zoltan-reg-file.xml opened file in Notepad++ 

Spend some times looking at the file and try to follow its structure based on the Concept section above. The most important part of this file is at the bottom of the file. Thus scroll down to learn about the exact data that we’ll test.

     Figure 2: The interesting data is colored BLACK. That is what you are trying to match. 

Create the value in your registry as shown below.

      Figure 3: Create a registry value that we'll match with OVAL. This will represent the malicious entry 

Locate the version for your notepad.exe file in c:\windows\system32 by looking at the properties of the executable.

                                          Figure 4: Look up the version number of the executable you want to match 

Note: On Windows 8.1 you might see an updated version of notepad. Make sure to use the version number your notepad shows and not the value that is provided here. You need to match your version of notepad version as a suspected malware. 

Thus, in my version, it shows 5.1.2600.5512, so that is the value I have to set in the version of the file state in my definition file line number 82.

If you have done everything correctly, you should be able to run ovaldi with no errors and have a report generated based on the two signatures we just created.

Download md5sum from http://www.etree.org/md5com.html and place it in the same directory as your ovaldi. If you feel comfortable with computers and want to keep md5sum on your system, you can also just copy it to c:\windows\system32 directory so it will be available from any directory on the command line. 

Generate the MD5 value for the definition file zoltan-reg-file.xml since you will need the md5 value to run ovaldi. ( If you do not wish to validate your definition file with md5, you can just use the –m option when you run ovaldi and it will not complain about not having the MD5 value. Since you are in security, you should not think about convenience, but what it the correct way to run it. I would suggest validating every time. ) 

C:\Program Files\OVAL\ovaldi-5.8.2>md5sum zoltan-reg-file.xml 
bbd2cfc9bd31e175716bd0f96d2be943 *zoltan-reg-file.xml 

Then, run the ovaldi with your custom definition file.

C:\Program Files\OVAL\ovaldi-5.8.2>ovaldi.exe -o zoltan-reg-file.xml bbd2cfc9bd31e175716bd0f96d2be943 
---------------------------------------------------- 
OVAL Definition Interpreter 
Version: 5.8 Build: 2 
Build date: Oct 13 2010 20:30:22 
Copyright (c) 2002-2010 - The MITRE Corporation 
---------------------------------------------------- 
Start Time: Fri Feb 18 14:04:55 2011 
** verifying the MD5 hash of 'zoltan-reg-file.xml' file 
** parsing zoltan-reg-file.xml file. 
 - validating xml schema. 
** checking schema version 
- Schema version - 5.8 
** skipping Schematron validation 
** creating a new OVAL System Characteristics file. 
** gathering data for the OVAL definitions. 
 Collecting object: FINISHED 
** saving data model to system-characteristics.xml. 
** running the OVAL Definition analysis. 
 Analyzing definition: FINISHED 
** applying directives to OVAL results. 
** OVAL definition results. 
 OVAL Id                             Result 
 ------------------------------------------------------- 
 oval:zoltan:def:1                     true 
 oval:zoltan:def:2                     true 
 ------------------------------------------------------- 
** finished evaluating OVAL definitions. 
** saving OVAL results to results.xml. 
** running OVAL Results xsl: xml\results_to_html.xsl. 
---------------------------------------------------- 

Look at the HTML reports. TRUE means, the vulnerability exist since we have matched the values we specified.

     Figure 5: Both rules have matched, the vulnerability exists for both threats. 

If everything worked for you the first time, congratulation. You should still create controlled problems to see how the results change and see what it means to get a failed result.

I changed the value in the registry for the example as it shown below.

                                           Figure 6: Change the value so it will not match the rule in the definition file 

I ran ovaldi again and I should get a failed value in the report that would mean the vulnerability does not exist on this system.

 OVAL Id                                    Result 
 ------------------------------------------------------- 
 oval:zoltan:def:2                            true 
 oval:zoltan:def:1                            false 
 ------------------------------------------------------- 

Now, the resulting report should look like this.

      Figure 7: After the registry value was changed, the OVAL report shows no vulnerability for the registry, but the 
      executable vulnerability still exists 

In this case, it would mean that I do not have the vulnerability on my system in the registry, but I still run a notepad version that is matching the rule that is defined based on the version number of notepad.exe in c:\windows\system32 directory.

Add References

You can also add references within every definition and if you are identifying policy violations, then you can just point the reference to the corporate website where the policy is listed or a central depository like in this example:

<reference source="CVE" ref_id="CVE-2010-2993" ref_url="http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE -2010-2993"/>

<reference source="RLC" ref_id="RLC-SP2011-001" ref_url="http://Richlandcollege.edu/forensics"/>

This is an en example that you can use to add reference to any definition. 

Figure 8: Reference added to the definition to easily access information about the vulnerability 

After you add the reference to your definition file, you will have to regenerate the MD5 hash before running ovaldi again. This time, you should see the reference link that you can just click and you’ll be redirected to the specified website. 

     Figure 9: The report shows the reference ID with a link


XML file contents ( zoltan-reg-file.xml )

<?xml version="1.0" encoding="UTF-8"?>
<oval_definitions xsi:schemaLocation="http://oval.mitre.org/XMLSchema/oval-definitions-5 oval-definitions-schema.xsd http://oval.mitre.org/XMLSchema/oval-definitions-5#windows windows-definitions-schema.xsd http://oval.mitre.org/XMLSchema/oval-definitions-5#independent independent-definitions-schema.xsd http://oval.mitre.org/XMLSchema/oval-common-5 oval-common-schema.xsd" xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:oval="http://oval.mitre.org/XMLSchema/oval-common-5" xmlns:oval-def="http://oval.mitre.org/XMLSchema/oval-definitions-5">
  <generator>
    <oval:product_name>Zoltan's Incident Response</oval:product_name>
    <oval:schema_version>5.8</oval:schema_version>
    <oval:timestamp>2011-02-18T04:34:07.876-05:00</oval:timestamp>
  </generator>

<definitions>

<definition id="oval:zoltan:def:1" version="1" class="vulnerability">

<metadata>
<title>Example Testing For Registry Value</title>
<description>
This definition is used to introduce the OVAL
Language to individuals interested in writing
OVAL Content to test for REGISTRY values.
</description>
</metadata>

<criteria>
<criterion test_ref="oval:zoltan:tst:1" comment="the value of the registry key equals Hello World"/>
</criteria>

</definition>

<definition id="oval:zoltan:def:2" version="1" class="vulnerability">

<metadata>
<title>Example Testing for File</title>
<description>
This definition is used to introduce the OVAL
Language to individuals interested in writing
OVAL Content to test for FILES.
</description>
</metadata>

<criteria>
<criterion test_ref="oval:zoltan:tst:2" comment="the value of the file is notepad"/>
</criteria>

</definition>
</definitions>

<tests>

<registry_test id="oval:zoltan:tst:1" version="1" comment="The value of the registry key must be Hello World" xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#windows" check="all">
<object object_ref="oval:zoltan:obj:1"/>
<state state_ref="oval:zoltan:ste:1"/>
</registry_test>

   
<file_test id="oval:zoltan:tst:2" version="1" comment="Test for file version notepad.exe" check_existence="at_least_one_exists" check="all" xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#windows">
      <object object_ref="oval:zoltan:obj:2"/>
      <state state_ref="oval:zoltan:ste:2"/>
    </file_test>

</tests>

<objects>
<registry_object id="oval:zoltan:obj:1" version="1" xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#windows">
<hive>HKEY_LOCAL_MACHINE</hive>
<key>SOFTWARE\oval</key>
<name>example</name>
</registry_object>
<file_object id="oval:zoltan:obj:2" version="1" xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#windows">
<path var_ref="oval:zoltan:var:1" var_check="all"/>
<filename>notepad.exe</filename>
</file_object>
<registry_object id="oval:zoltan:obj:3" version="1" comment="This registry key identifies the system root." xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#windows">
<hive>HKEY_LOCAL_MACHINE</hive>
<key>SOFTWARE\Microsoft\Windows NT\CurrentVersion</key>
<name>SystemRoot</name>
</registry_object>
</objects>
<states>
<registry_state id="oval:zoltan:ste:1" version="1" comment="The registry key matches with version of the Wireshark 1.2.0 to 1.2.9" xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#windows">
<value>Hello World</value>
</registry_state>
<file_state id="oval:zoltan:ste:2" version="1" xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#windows">
<version datatype="version" operation="equals">6.3.9600.16383</version>
</file_state>
</states>
<variables>
<local_variable id="oval:zoltan:var:1" version="1" comment="Windows system 32 directory" datatype="string">
<concat>
<object_component item_field="value" object_ref="oval:zoltan:obj:3"/>
<literal_component>\System32</literal_component>
</concat>
</local_variable>
</variables>


</oval_definitions>

PowerShell performance testing

We all remember LogParser as an amazing little tool that "could" and after Vista, it could not.  So, what other alternatives we have these days?  That question triggered me to look into "wevtutil" since its documentation talks about the possibilities of running queries, exporting, archiving, and clearing logs.  

We can clear a single log by wevtutil cl Security, but we have many logs on systems to manage.  It is also possible in an event of system compromise to clear all the logs by the intruder.  So, how would one clear all logs in a simple and easy step?  Use ProwerShell most likely or a DOS for loop might suffice.

I wanted to look at PowerShell and its capability to do this task.  It can actually do this very easily by a single line of command:

wevtutil el | Foreach-Object {wevtutil cl "$_"}

The el parameter enumerates all logs on the system, so that output can be used to send each log name to wevtutil to clear each log.  

It is nice, but I also wanted to know how many of these logs I have on my system.  So, PowerShell was the natural choice for this task.  A few minutes of search on the Internet revealed that there are many ways to count objects in PowerShell.  This is where the true investigator gene triggered in my head and wanted to find out their differences.  

Digital Forensics is partially a field of inductive reasoning where each step of the reasoning needs to be reliable and scientifically tested in order to come to a reliable conclusion.  It will never be as reliable as deductive logic, but well tested methodologies can make it as reliable as the latest knowledge allows us.  


One benefit of education is the ability to test tools and make heuristic decisions when using them.  In some cases, we might even determine a suspect education level and sophistication of attacks that we might be facing if we could capture and analyze the code he/she uses in reconnaissance.  After all, we need to connect a human to a technical action in order to call this field a science.

But that is enough.

I decided to test three methods of counting.  I needed a large set of objects to iterate through in order to see a trend.  I decided to count the number directory entries on my system drive.  That should be large enough to see any emerging trend.  I have stopped all unnecessary services on my system to create  stable environment and to establish a baseline.  I will be monitoring one process, but wanted to make sure nothing interferes with my system while testing.  I ran PowerShell as an administrator and ran each command three times with a little pause in between each command.  

I was not surprised about the similarity of the I/O operations since in each case the directory objects needed to be accessed, but the memory utilization did surprise me in the third case.  It seems like the third method gathers all file objects before counting them at the end.  The first two methods seem to pick up an object increment a counter and then drop the object while the third method picks up all objects and hangs on to them before determine their count.

The commands I tested:

I.   Get-ChildItem -recurse -EA SilentlyContinue|measure

II.  Get-ChildItem -recurse -EA SilentlyContinue|foreach-object { $count++}

III. (Get-ChildItem -recurse -EA SilentlyContinue).count

      $count

Note: the -EA SilentlyContinue parameter was used to suppress error messages since there are some directories that not even an administrator can access.  I first ran a few tests without this parameter, but the errors were annoying, so I decided to add the parameter later on.  It is the part of the scientific method to evaluate and to adjust procedures along the way.

It might never be interesting in an actual investigation, but in order to develop a pattern recognition skill, this type of exercises can develop investigators that are not just logical thinkers, but problem solvers who will not just pick up the first tool he/she finds and treat it like an IT person would just to get the job done, but ensures the best and most reliable methods in investigations.  By the way, all three tools did find the same amount of directory entries and they did compete the task in the same amount of time.  

References:
http://technet.microsoft.com/en-us/library/cc732848.aspx
http://technet.microsoft.com/en-us/library/ee176841.aspx

Android phone Thumbdata - Part 3

In this blog, I wanted to look further into the data storage on Android phone and look at the files in the DCIM\.thumbnails folder. 

 

 The first files I wanted to look at was the .thumbdata3-.... files.  The first one had a modification time of the time I took the last picture with the phone and the other the modification time of the time I recorded the last video.

Further examination showed that the first file contains 9 thumbnail images starting with the first image I took and each record is 10000 bytes long, so just by looking at the file size, we can see how many image thumbnails each one of these files contain.  Thus, the second thumbdata file is used to keep track of the thumbnails related to the videos in DCIM\Camera folder.  Each thumbnail is stored as a JPEG image that can be manually carved out by selecting the range between the signatures FFD8FF - FFD9.  

The other files that were interesting for further analysis were the image_last_thumb and video_last_thumb files.  What was most interesting to see the modification time that showed the time when the last image was taken and not the last image that was viewed.  Same was the case with the video_last_thumb, it showed the modification time of the last video recording.  

11/30/2012

12:48am formatted using on the phone utility

12:58-12:59am took pictures 1-9

1:00 am pencil video was created

1:01am CD video was created

Thus, I wanted to see what these files contained since the file mane and modification times were not reflecting what they suggested.

The last events that actually happened on the phone was that I viewed one image and the first video.

1:19am image 6 viewed

1:20am  pencil video viewed

These two files begun with the following text:

                 image_last_thumb

                          'content://media/external/images/media/9‰PNG

                 video_last_thumb

                           .&content://media/external/video/media/2‰PNG

Thus, these files are keeping the last image and last video that was added to the storage media and not the files that were last viewed.  These are the imaged that show up when a user enters the Gallery application.  

The signature of the PNG chunk is clearly visible in these files, so simply cutting the leading text and splicing the correct files header for a PNG file will allow us to see the actual images.  So, I have created a PNG file and copied its 12 byte header into each one of these files.

The resulting files could be viewed in any image viewer and confirmed the content was matching the path and file name stored in these files and these files contained the last multimedia file added to the storage device and not the last image viewed.  It is interesting why Google chose to save the thumbnail images in JPEG format and the last thumb images as PNG. 

Conclusion

Always verify your hypothesis with manual methods instead of relying third party tools.  The ability to verify findings in this manner gives you more confidence and develops your analytical skills.