Move Microsoft Sentinel Logs to Long-Term Storage | StarWind Blog (2024)

FAQs

Which methods can you use to send Azure Sentinel logs to long term storage? ›

KQL Function: Kusto Query Language (KQL) is used to query and analyze data in Azure Sentinel. The KQL function can be used to create a query that retrieves logs from a workspace and exports them to a storage account or blob container.

By default, a Log Analytics workspace has a retention period of 30 days.

After you enable Microsoft Sentinel on a Log Analytics workspace: You can retain all data ingested into the workspace at no charge for the first 90 days. Retention beyond 90 days is charged per the standard Log Analytics retention prices.

Archive logs to an Azure storage account

Sign in to the Azure portal. Select Azure Active Directory > Monitoring > Audit logs. Select Export Data Settings.

AzCopy is a command-line tool for copying data to or from Azure Blob storage, Azure Files, and Azure Table storage, by using simple commands. The commands are designed for optimal performance. Using AzCopy, you can either copy data between a file system and a storage account, or between storage accounts.

SOX: The Sarbanes-Oxley Act (SOX) concerns corporations active in the United States and requires them to keep audit logs for seven years. CISP: The Cardholder Information Security Program (CISP) pertains to all ecommerce corporations and requires them to keep their logs for a minimum of six months.

By default, logs ingested into Microsoft Sentinel are stored in Azure Monitor Log Analytics.

As a baseline, most organizations keep audit logs, IDS logs and firewall logs for at least two months. On the other hand, various laws and regulations require businesses to keep logs for durations varying between six months and seven years. Below you can find some of those regulations and required durations.

If a paid subscription ends or is terminated, Microsoft retains customer data stored in Microsoft 365 in a limited-function account for 90 days to enable the subscriber to extract the data. After the 90-day retention period ends, Microsoft disables the account and deletes the customer data.

What are the 4 primary capabilities of Microsoft Sentinel? ›

With Microsoft Sentinel, you get a single solution for attack detection, threat visibility, proactive hunting, and threat response.

Watchlists can only be referenced from within the same workspace. Cross-workspace and/or Lighthouse scenarios are currently not supported. Local file uploads are currently limited to files of up to 3.8 MB in size.

We recommend that you use Azure Storage logs in Azure Monitor instead of Storage Analytics logs. To learn more, see any of the following articles: Monitoring Azure Blob Storage. Monitoring Azure Files.

Go to >App Service Logs> Enable Application Logging (Blob) and select the desired Blob Storage and the Container and select the Level of logging.

Copy a container to another storage account by using the azcopy copy command. This example encloses path arguments with single quotes (''). Use single quotes in all command shells except for the Windows Command Shell (cmd.exe).

Move data to the new storage account

AzCopy is the preferred tool to move your data over. It's optimized for performance. One way that it's faster, is that data is copied directly between storage servers, so AzCopy doesn't use the network bandwidth of your computer.

Azure Storage blobs

Azure Storage is the most ubiquitous storage solution Azure provides, due to the number of services and tools that can be used with it. There are various Azure Storage services you can use to store data. The most flexible option for storing blobs from many data sources is Blob storage.

How big is too big for a log file? ›

The maximum size for a log file is two terabytes. Enable Autogrowth: Autogrowth enables the SQL Server to expand the size of database files when they run out of space.

Specify the maximum log file size Policy particular policy determines the upper limit of the log file size in kilobytes. By enabling this Policy, you can set the maximum size of the log file between 1 megabyte (1024 kilobytes) and 2 terabytes (2147483647 kilobytes), with increments of kilobytes.

Centralize Your Logs

That's why the most important log retention best practice is to archive logs into a central repository, such as a security information and event management (SIEM) platform. A SIEM not only collects logs, but it correlates logs and other security-related documentation for analysis.

SIEM tools collect and aggregate log data from across the IT infrastructure into a centralized platform where it can be reviewed by security analysts. They also deliver SIM features, such as automation and alerts, and the correlative capabilities of SEC tools.

The device's built-in Syslog daemon collects local events of the specified types, and forwards the events locally to the agent. The agent streams the events to your Log Analytics workspace. After successful configuration, the data appears in the Log Analytics Syslog table.

In Windows, the event logs are stored in the C:\WINDOWS\system32\config\ folder. They are created for each system access, operating system blip, security modification, hardware malfunction and driver issue.

Passwords, IP addresses and network information (MAC address, host name, etc.)

The standard mandates that audit logs be retained for at least one year. Ninety days of PCI audit logs must also be available for immediate analysis.

For most Microsoft products, data retention is 30 days.

You can retain audit logs for up to 10 years.

How long does Microsoft keep activity history? ›

The Recent activity page shows you when and where you've used your Microsoft account within the last 30 days. You can expand any listed activity to see location details and find out how the account was accessed—using a web browser, phone, or another method.

In MS Access, all information is saved in one file which has a hard restriction – it cannot be larger than 2GB. Consequently, Access is not ideal for handling large databases with tens of thousands of rows and attached information like images or files.

Compared to Splunk, it is easier to deploy, and has superior artificial intelligence. In addition, Microsoft Sentinel's price is more attractive than Splunk's. To learn more, read our detailed Microsoft Sentinel vs. Splunk Enterprise Security Report (Updated: May 2023).

Microsoft Sentinel aggregates data from all sources, including users, applications, servers, and devices running on premises or in any cloud, letting you reason over millions of records in a few seconds. It includes built-in connectors for easy onboarding of popular security solutions.

Azure Sentinel, now known as Microsoft Sentinel, centralizes your threat collection, detection, response, and investigation efforts. It provides threat intelligence and intelligent security analytic capabilities that facilitate threat visibility, alert detection, threat response, and proactive hunting.

By default, a Log Analytics workspace has a retention period of 30 days. Retention is calculated on the ingestion date for data, so if a workspace uses the default retention period, it means that Azure removes data from the workspace 30 days after its ingestion.

In your Log Analytics workspace, clear the inherit the workspace setting so the interactive retention period is fixed to 30 days.

Data retention and archived logs costs

After you enable Microsoft Sentinel on a Log Analytics workspace: You can retain all data ingested into the workspace at no charge for the first 90 days. Retention beyond 90 days is charged per the standard Log Analytics retention prices.

text or comma-delimited format.

By default, no version of Windows creates a log of files that have been copied, whether to/from USB drives or anywhere else.

Does Windows log everything? ›

If you're like most Windows 10 users, you might not know that your computer keeps logs of everything that goes in it. Furthermore, you can use these logs to troubleshoot any security issues on your Windows PC. Simply put, system and security logs are records of events and activities on your PC.

A data retention period of 90 days means that developers and security teams will have access to a rolling 90-day window of indexed log data for analytics purposes - that's your data retention window.

Event log files can be saved as event files (*. evt), text files (*. txt). or comma-delimited text files (*. txt).

Importing Event Log File. Select the Settings tab. In the System Settings section, click the Imported Log File link. Select the Event Log Imports / Application Log Imports tab, and click the Import Log File link on the right side, to import a new event/application log file.

Site administrators have the ability to export processed log files into a single compressed file that contains the daily logs for the specified date range. Logs can be exported in any log file format, regardless of the original web server that initially created the log files.

We recommend that you use Azure Storage logs in Azure Monitor instead of Storage Analytics logs.

In Microsoft Sentinel, select Data connectors from the navigation menu. From the data connectors gallery, select Azure Active Directory and then select Open connector page. Mark the check boxes next to the log types you want to stream into Microsoft Sentinel (see above), and select Connect.

Storage Explorer can connect to a storage account using the storage account's name and key. You can find your account keys in the Azure portal. Open your storage account page and select Settings > Access keys. In the Select Resource panel of the Connect to Azure Storage dialog, select Storage account.

What is the most efficient way to store logs? ›

Use Pallets where possible

Preferably, logs should be placed on wooden pallets as these keep them off the ground and provide a free flow of air underneath; the ideal height of the wood stack (including the pallet) should be no more than 3ft (1m) as the logs can become unstable if piled too high.

Firewood is best stored outside. It should be stored neatly, with the outside of the wood exposed to the air. If possible, you should place the wood on top of plastic sheeting or in a wooden log store. Avoid tree cover if possible and don't leave the logs in a heap.

By default, logs ingested into Microsoft Sentinel are stored in Azure Monitor Log Analytics.

This blog will use both Microsoft Sentinel and Azure Sentinel, but for the sake of clarity, both terms refer to the same product. Azure Sentinel is a cloud-based security information and event management (SIEM) solution that helps you detect, investigate, and respond to threats across your entire organization.

On the Log Analytics workspace menu in the Azure portal, select Data Export under the Settings section to view all export rules in the workspace.