If you’re like me, then working in IT means you also assume Tech Support duties for friends, family, and those distant relatives that only seem to call when they’ve got a problem.

I just clicked on this link, and my computer is doing something weird. I think my PC has a virus, what do I do?

When it’s just a single computer, the answer is simple, contain and validate the rouge software is removed, install an AV solution, change their passwords, enable MFA, and provide some education on what to look out for next time.

But now imagine you’re an organisation building a new application, or are moving applications to the cloud. Are you simply performing a lift-and-shift or are you planning to make use of cloud native services? Where are you going to store your data, specifically user uploaded files? Object Storage was built specifically to solve the challenges of how to store unstructured data in the cloud.

However, there is a catch. If you were previously storing files on a server file system, then it’s likely you were also running an anti-virus / anti-malware solution to identify malicious files. With Object Storage the underlying file system is transparent, so you can’t install AV, yet many compliance requirements still state “Uploaded files must be scanned for viruses and malware”.

The Shared Responsibility Model

I’ve heard many times that “We’re in the cloud, so my provider takes care of that right? Unfortunately not. Cloud Providers are responsible only for the security of their platform, you are responsible for what you put in the cloud. If you’re not familiar with the concept, take a look at: https://www.oracle.com/a/ocom/docs/cloud/shared-responsibility-model-wp.pdf

I’m using Object Storage and need to scan for malware, what are my options?

• DIY, and build scanning integration capabilities into each application.
• Use a 3rd Party Cloud Security Posture Management tool that has AV / Malware scanning capabilities.
• Or don’t use Object Storage, store files on a file system where an AV / Malware solution is installed. Unless you have a specific requirement to interact with the underlying file system I wouldn’t recommend this for a range of reasons, but it is still an option.

Each of these options has its merits. In any case they require time, and some may end up costing more than any benefits you might realise. Saying that, I’ve also seen first hand what happens in the absence of any anti-malware controls, and object storage buckets being full with malware.

It isn’t all doom and gloom however! If you’re using Oracle Cloud Infrastructure I’m going to show you how you can easily achieve the requirement of checking uploaded files using OCI native services and the https://virustotal.com/ API.

It’s important to mention that I’m using a community API key from https://virustotal.com/ for demo purposes only. And you could easily adapt this approach to use any AV / Malware scanning solution that has an API.

AV / Malware Scanning Considerations

Before I jump into showing you how to configure this in your own OCI Tenancy, there are some things that must be considered first. These considerations would also apply if you were wanting to DIY a solution, or use a Cloud Security Posture Management Tool.

• Data Sovereignty. Do you need to keep data within a specific set of geographic regions? Your chosen cloud anti-malware solution may not store data in your geographic area, or they may replicate your data across the globe.
• Local Laws & Regulations. What data resides in your files? You may not want files containing PII leaving your environment, or accessible by 3rd Parties.
• 3rd Party Risks. If you are using a cloud based anti-malware solution, you’ll want to ensure they have adequate controls to protect the privacy of your data.
• AV / Malware Licensing & Infrastructure. Do you want to purchase, install and manage the solution, integrations, and support the underlying infrastructure?
• In-band vs Out-of-band scanning. If users can upload files, specifically large files, the time required to scan may have a negative impact on user experience if they have to wait. Similarly if you scan post upload you’ll need to consider how to handle broken file links when a file is quarantined.
• Adequate Coverage. Files can get into object storage many different ways including web forms, API’s, FTP, WEBDAV etc. Whatever solution you choose needs to ensure all files are scanned.
• Performance & Scalability. You’ll need to plan how to handle large files, and potentially large numbers of files. If you’re doing in-band scanning, then there is the possibility of users experiencing a Denial of Service if you don’t have adequate capacity.
• Fail open vs closed. In the case there’s scan failure, or the scanning service is unavailable, do you want to reject the file, or let it through?
• Operating Model. If you’re doing out-of-band scanning, how will you support reviewing and responding to files identified as malicious.

Checking for Malware in Oracle Cloud Infrastructure

The image above illustrates the architecture of my proposed solution. It comprises of:

• A Compartment, Virtual Cloud Network, Public Subnet, and Internet Gateway. This will likely already exist in your own environment.
• A custom Ruby OCI Function called scan-upload.
• One or more buckets where files are uploaded, in my diagram these are called bucket-1 and bucket-2.
• An Event rule, that triggers the scan-upload function when an object is uploaded to a Bucket.
• An external API provider returning metadata for known malicious MD5 file hashes. For this demo I’ve chosen Virus Total.
• A quarantine Bucket where files that meet the quarantine criteria are moved.
• A Custom Log called scan-results where JSON file scan results are stored.
• A Service Connector that pushes logs from the scan-results Custom Log to Object Storage.
• An Object Collection Rule, Custom Parser and Dashboard in Logging Analytics to view file upload activity and files flagged as malicious.

The benefits of this approach are:

• The Event Rule will fire for every object uploaded regardless of which Bucket. You can exclude certain Buckets by specifying then in the Function configuration.
• You can deploy this in more regions, ensuring data sovereignty.
• Only the file’s MD5 Hash is send to the 3rd party, removing PII and 3rd party risks.
• The function makes an outbound HTTPS call only, no inbound rules are required.
• Scanning is performed out-of-band / post upload, meaning we’re not impacting the user experience.
• It’s fast, as we’re only checking the MD5 hash.
• It’s scalable as OCI functions scale automatically.
• You can easily extend the existing Function logic to handle your specific rejection workflows e.g. Updating a file link in your database.
• You can view real-time insights in Logging Analytics.

For this demo I’ve limited the scope to a specific compartment and region, however you could easily expand this approach to multiple regions and compartments.

Setup & Configuration

Before we begin, you’ll need a Compartment, Virtual Cloud Network, Public Subnet, and Internet Gateway. If required you can create the networking components via the VCN Wizard.

You’ll also need to install Docker, and the Fn Project. On OSX installing the Fn Project is easy with Homebrew:

$ brew install fn

To push the function to the OCI container repository you’ll also need an Auth Token which can be generated in OCI Identity by selecting your user, and clicking “Auth Tokens”.

Download the function source code from Github https://github.com/scotti-fletcher/oci-av-function. As always I recommend inspecting any source code that you download from the Internet for security issues.

Now we’ll create a Functions application. An application is just a logical grouping of functions, and in this demo we only have one application called “red-thunder-demo” and one function called “scan-upload”.

After creating the application, follow the “Getting Started” instructions for “Local setup”. The instructions displayed will be unique for your OCI environment, however I’ll explain the steps as not all are required for this demo.

In terminal, cd into the scan-upload directory containing the function files:

scott@scott-mac ~ % cd ~/Desktop/oci-av-function-main/scan-upload 
scott@scott-mac scan-upload % ls
Gemfile		func.rb		func.yaml

Create a context for this compartment and select it for use. As my compartment is called “red-thunder-demo” you see it in the next statement.

$ fn create context red-thunder-demo --provider oracle
$ fn use context red-thunder-demo

Update the context with the compartment OCID where you created the application and the Oracle Functions API URL.

$ fn update context oracle.compartment-id [your compartment ocid]
$ fn update context api-url https://functions.ap-sydney-1.oraclecloud.com

Create a repository where your function container image will be stored.

$ fn update context registry syd.ocir.io/[your namespace]/red-thunder-demo

Log into the OCI container registry.

$ docker login -u '[your namespace]/oracleidentitycloudservice/scott.fletcher@oracle.com' syd.ocir.io

You should see a “Login Successful” message. You can now build and deploy your application. The first time you do this it might take a few minutes:

scott@scott-mac scan-upload % fn deploy --app red-thunder-demo
Deploying scan-upload to app: red-thunder-demo
Bumped to version 1.0.1
Using Container engine docker
Building image syd.ocir.io/abcd/red-thunder-demo/scan-upload:1.0.1 ...................................................
Parts:  [syd.ocir.io abcd red-thunder-demo scan-upload:1.0.1]
Using Container engine docker to push
Pushing syd.ocir.io/abcd/red-thunder-demo/scan-upload:1.0.1 to docker registry...The push refers to repository [syd.ocir.io/abcd/red-thunder-demo/scan-upload]
cd394e6f57be: Pushed 
71adf80122ed: Pushed 
dcec050cc620: Pushed 
3668420c8d32: Mounted from abcd/test-av-setup/scan-upload 
5e4a9dac9f74: Mounted from abcd/test-av-setup/scan-upload 
dedbd9e80d19: Mounted from abcd/test-av-setup/scan-upload 
7c1c8ad8e410: Mounted from abcd/test-av-setup/scan-upload 
1.0.1: digest: sha256:769bbe97af259c710033b59e494c9765413a11e7cb5eab19fade525e293c82ee size: 1783
Updating function scan-upload using image syd.ocir.io/abcd/red-thunder-demo/scan-upload:1.0.1...
Successfully created function: scan-upload with syd.ocir.io/abcd/red-thunder-demo/scan-upload:1.0.1

Once created, you should be able to see the function in the OCI console under applications and also in the Container Registry.

Now we need to create a Log Group to hold our Function specific logs and scan results. Create a log group, I’ve named mine “av-function-log-group”:

In logs, click “Create custom log” and call it “scan-results”.

When prompted select “Add configuration later” and click create. You should see the log “scan-results” in the “av-function-log-group”:

Click the “scan-results” log, and note down the OCID as we’ll need this later.

Now we need to create the following Buckets:

When creating the buckets, I’m using the default options however you need to ensure the “Emit Object Events” checkbox is selected on bucket-1 and bucket-2. If you have other buckets that you wish to scan you’ll need to update them too. By clicking “Emit Object Events” when an object is uploaded the Events Service rule will trigger our function.
Note: We don’t “Emit Object Events” for our quarantine bucket, otherwise the function will run twice for the same file.

Next we need to create an Event Service rule that triggers our function:

Now we need to create a Service-Connector to push the scan-results logs into a bucket so they can be collected and ingested by Logging Analytics:

Note you will need to “Create default policy” as prompted. When you click create this will also enable the “Emit Object Events” on the scan-results bucket as it’s required for the Service Connector to run.

Now we’ll go back to our Function, and update the required configuration items. Configuration items are just environment variables that are made accessible to the function:

You can configure the following items:

• VT_API_KEY: This is your Virus Total API Key.
• QUARANTINE_BUCKET_NAME: This is the name of the bucket where you want quarantined files to be moved. In this demo we created a bucket called “quarantine” for this purpose.
• EXCLUDE_BUCKETS: If you have any buckets that you don’t want scanned you can add them here as a comma separated list (with no spaces). Because we don’t want our logs in the scan-results bucket scanned (as there’s no point) I’ve added it here.
• QUARANTINE_THRESHOLD: Virus Total uses a number of different scan engines to provide great coverage. My value of 1 means if 1% of the scan engines report the file as malicious, then I want the function to move the file to the quarantine bucket. I suggest a low value here.
• DELETE_THRESHOLD: My value of 50 means if 50% of the scan engines report the file as malicious, then I want the function to delete the file. Depending on your specific use case you might want to increase or decrease this value.
• OCI_LOG_OCID: This is the OCID of the scan-results custom log you created earlier.
• QUARANTINE_BUCKET_REGION: This is the region where the bucket called “quarantine” is located. I’ve entered ap-sydney-1 as this is where I created the bucket.
• ON_ERROR: As I mentioned earlier, you need to consider what to do if there’s a scan error, network connectivity failure, or API failure (like exceeding your Virus Total API limits). Depending on your use case you might want to delete the object, quarantine the object, or fail open. You can choose to enter “QUARANTINE”, “DELETE”, or remove this configuration item altogether and fail open if an error occurs.

Functions also emit logs, and it’s useful to see them for debugging purposes. To enable these logs, click “Enable Log”:

Before we test our function, we need to create a Dynamic Group, and Policies to allow our function to operate in our environment. Depending on your use-case and where your object storage buckets are, you may need to adjust the policies. My examples below are scoped to the “red-thunder-demo” compartment.

Create a Dynamic Group that defines the Function and compartment:

Create a new policy, with the following policy statements. If you’ve chosen different Dynamic Group and Compartment names, you will need to update the policy statements accordingly:

• “allow dynamic-group red-thunder-demo-scan-upload-dynamic-group to manage objects in compartment red-thunder-demo”. This allows the function defined by the Dynamic Group to manage Objects.
• “allow dynamic-group red-thunder-demo-scan-upload-dynamic-group to use log-content in compartment red-thunder-demo”. This allows the function to write to the custom log that we created to hold our scan results.The last two statements:
• “allow service faas to {KEY_READ} in compartment red-thunder-demo where request.operation=’GetKeyVersion’”
• allow service faas to {KEY_VERIFY} in compartment red-thunder-demo where request.operation=’Verify’allow the functions service to read the key used to encrypt and decrypt the files stored in Object Storage.

You should now have two policies in your compartment:

At this point the function should be installed and configured correctly. Before we continue I feel it’s important to mention that handling malware does come with risks and you should take appropriate steps to protect yourself and your organisation from inadvertently executing malware. If you’re not familiar with safe handling practises, consult someone who is, and I’d also recommend using a sandbox environment for the next parts of this demo.

I’ve uploaded two ZIP files containing malware. I chose ZIP files specifically so the files contained are not immediately executable. I also know these files are flagged by Virus Total. I’ve uploaded them to bucket-1 and bucket-2. Once uploaded looking at the function invocation logs I can see the function has run. One object is quarantined because the confidence level is 5%, and the other is deleted outright because the confidence is 80%, which is above my configured value of 50:

At this point, we’ve confirmed the scan-upload function works as expected. Great! However looking at these logs isn’t very exciting so I’m going to show you how you can visualise these results with Logging Analytics.

If you’ve been following this guide then you might have noticed we’ve already set ourselves up to collect and ingest logs into Logging Analytics. We did this by creating a scan-results bucket and a Service Connector. If you’ve uploaded some malware, after about 10 minutes you should see some logs in the scan-results bucket:

In Logging Analytics create a Log Group to hold our scan-results logs. I’ve called mine “malware-scan-results”:

Download the Logging Analytics Log Source & Parser configuration if you haven’t retrieved it from GitHub https://github.com/scotti-fletcher/oci-av-function/blob/main/malware-log-source.zip

Click “Import Configuration Content”, select the ZIP file and click Import:

Now we need to create an Object Collection Rule that enables collection of the logs from our scan-results bucket and loads them into our malware-scan-results Log Group.

$ oci log-analytics  object-collection-rule create --name malware-collection-rule --compartment-id [Your Compartment OCID] --os-namespace [Your Object Storage Namespace]  --os-bucket-name scan-results --log-group-id [The OCID of the malware-scan-results Log Group] --log-source-name malware-log-source --namespace-name [Your Object Storage Namespace] --collection-type HISTORIC_LIVE --poll-since BEGINNING

The CLI should return a JSON response, indicating your rule creation succeeded. You should also be able to see another Event Rule created:

Download the Logging Analytics Dashboard if you haven’t already from GitHub https://github.com/scotti-fletcher/oci-av-function/blob/main/malware-dashboard.json . You’ll need to find and replace all values of “ENTER_COMPARTMENT_ID” with the OCID of the compartment where you want to create the dashboard.

Using the CLI run the following command. You’ll also need to update the URL to the endpoint for the region where you want the Dashboard to be created:

$ oci raw-request --target-uri https://managementdashboard.ap-sydney-1.oci.oraclecloud.com/20200901/managementDashboards/actions/importDashboard --http-method POST --request-body file://malware-dashboard.json

You should then be able to view your dashboard. If you don’t see any data, make sure your scan-results bucket has logs in it, and you have selected the correct compartment from the drop down list:

The Dashboard I’ve created shows some useful information, such as the volume of files scanned, the scan results, the actions taken, the threat names, in which buckets your malware is located, and information about the files flagged as malicious.

Some Final Thoughts

Malware is one of those things we wish didn’t exist, however hopefully I’ve shown you how you can tackle the problem and perform a basic check of a file’s MD5 hash using OCI services.

From here, you could easily extend the Function’s logic to upload the files themselves to Virus Total, or submit them to your own anti-malware solution. You could also invoke other functions to drive your own custom workflows.

Happy Malware Scanning!