Jump to content

Log management: Difference between revisions

From Wikipedia, the free encyclopedia
Browse history interactively
← Previous edit
Content deleted Content added
VisualWikitext
Restored revision 968923758 by Tea2min (talk): Rm blog
m Leading chapter has been edited and fresh citation added
 
(12 intermediate revisions by 9 users not shown)
Line 4: Line 4:
}}
}}


'''Log management''' is the process for generating, transmitting, storing, accessing, and disposing of log data. A log data (or ''logs'') is composed of entries (records), and each entry contains information related to a specific event that occur within an organization’s computing assets, including physical and virtual platforms, networks, services, and cloud environments.<ref>NIST [https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-92r1.ipd.pdf SP 800-92r1], Cybersecurity Log Management Planning Guide</ref>
'''Log management''' ('''LM''') comprises an approach to dealing with large volumes of [[computer]]-generated [[data logging|log message]]s (also known as [[audit records]], [[audit trail]]s, [[event-logs]], etc.).


Log management generally covers:<ref>{{cite web|url=http://csrc.nist.gov/publications/nistpubs/800-92/SP800-92.pdf|title=SP 800-92, Guide to Computer Security Log Management|first1=Author: Karen Kent|last1=(NIST)|first2=Author: Murugiah Souppaya|last2=(NIST)|website=csrc.nist.gov}}</ref>
The process of log management generally breaks down into:<ref>{{cite report |title=Guide to Computer Security Log Management |id=NIST SP 800-92 |last1=Kent |first1=Karen |last2=Souppaya |first2=Murugiah |publisher=NIST |date=September 2006 |doi=10.6028/NIST.SP.800-92 |s2cid=221183642 |doi-access=free}}</ref>


* Log collection - a process of capturing actual data from log files, application standard output stream ([[Standard streams|stdout]]), network [[Network socket|socket]] and other sources.
* Log collection
* Logs aggregation (centralization) - a process of putting all the log data together in a single place for the sake of further analysis or/and retention.
* Centralized log aggregation
* Log storage and retention - a process of handling large volumes of log data according to corporate or regulatory policies (compliance).
* Long-term log storage and retention
* [[Log analysis]] - a process that helps operations and security team to handle system performance issues and security incidents
⚫
* [[Log rotation]]
* [[Log analysis]] (in real-time and in bulk after storage)
* Log search and reporting.


== Overview ==
== Overview ==
Line 24: Line 22:
* Normalization: logs are produced in multiple formats. The process of [[Normalization (statistics)|normalization]] is designed to provide a common output for analysis from diverse sources.
* Normalization: logs are produced in multiple formats. The process of [[Normalization (statistics)|normalization]] is designed to provide a common output for analysis from diverse sources.
* Velocity: The speed at which logs are produced from devices can make collection and aggregation difficult
* Velocity: The speed at which logs are produced from devices can make collection and aggregation difficult
* Veracity: Log events may not be accurate. This is especially problematic from systems that perform detection, such as [[Intrusion detection system|intrusion detection systems]].
* Veracity: Log events may not be accurate. This is especially problematic for systems that perform detection, such as [[Intrusion detection system|intrusion detection systems]].


Users and potential users of log management may purchase complete commercial tools or build their own log-management and intelligence tools, assembling the functionality from various [[open-source model|open-source]] components, or acquire (sub-)systems from commercial vendors. Log management is a complicated process and organizations often make mistakes while approaching it.<ref>{{cite web|url=http://www.docstoc.com/docs/19680768/Top-5-Log-Mistakes---Second-Edition|title=Top 5 Log Mistakes - Second Edition|work=Docstoc.com|accessdate=12 August 2015}}</ref>
Users and potential users of log management may purchase complete commercial tools or build their own log-management and intelligence tools, assembling the functionality from various [[open-source model|open-source]] components, or acquire (sub-)systems from commercial vendors. Log management is a complicated process and organizations often make mistakes while approaching it.<ref>{{cite web|url=http://www.docstoc.com/docs/19680768/Top-5-Log-Mistakes---Second-Edition|title=Top 5 Log Mistakes - Second Edition|work=Docstoc.com|accessdate=12 August 2015}}</ref>
Line 36: Line 34:
== Terminology ==
== Terminology ==


Suggestions were made{{by whom|date=October 2014}} to change the definition of logging. This change would keep matters both more pure and more easily maintainable:
Suggestions were made{{by whom|date=October 2014}} to change the definition of logging. This change would keep matters both purer and more easily maintainable:
* '''Logging''' would then be defined as all instantly discardable data on the technical process of an application or website, as it represents and processes data and user input.
* '''Logging''' would then be defined as all instantly discardable data on the technical process of an application or website, as it represents and processes data and user input.
Line 45: Line 43:
One view{{Citation needed|date=October 2007}} of assessing the maturity of an organization in terms of the deployment of log-management tools might use{{Or|date=October 2007}} successive levels such as:
One view{{Citation needed|date=October 2007}} of assessing the maturity of an organization in terms of the deployment of log-management tools might use{{Or|date=October 2007}} successive levels such as:


# in the initial stages, organizations use different log-analyzers for analyzing the logs in the devices on the security-perimeter. They aim to identify the patterns of attack on the perimeter infrastructure of the organization.
# in the initial stages, organizations use different log-analyzers for analyzing the logs in the devices on the security perimeter. They aim to identify the patterns of attack on the perimeter infrastructure of the organization.
# with increased use of integrated computing, organizations mandate logs to identify the access and usage of confidential data within the security-perimeter.
# with the increased use of integrated computing, organizations mandate logs to identify the access and usage of confidential data within the security perimeter.
# at the next level of maturity, the log analyzer can track and monitor the performance and availability of systems at the level of the [[Business|enterprise]] — especially of those information-assets whose availability organizations regard as vital.
# at the next level of maturity, the log analyzer can track and monitor the performance and availability of systems at the level of the [[Business|enterprise]] — especially of those information assets whose availability organizations regard as vital.
# organizations integrate the logs of various [[business]]-applications into an enterprise log manager for better [[value proposition]].
# organizations integrate the logs of various [[business]] applications into an enterprise log manager for a better [[value proposition]].
# organizations merge the physical-access monitoring and the logical-access monitoring into a single view.
# organizations merge the physical-access monitoring and the logical-access monitoring into a single view.


Line 58: Line 56:
*[[Data logging]]
*[[Data logging]]
*[[Log analysis]]
*[[Log analysis]]
⚫
*[[Log monitor]]
*[[Log management knowledge base]]
*[[Log management knowledge base]]
*[[Security information and event management]]
*[[Security information and event management]]
Line 69: Line 68:
* Chris MacKinnon: "LMI In The Enterprise". ''Processor'' November 18, 2005, Vol.27 Issue 46, page 33. Online at http://www.processor.com/editorial/article.asp?article=articles%2Fp2746%2F09p46%2F09p46.asp, retrieved 2007-09-10
* Chris MacKinnon: "LMI In The Enterprise". ''Processor'' November 18, 2005, Vol.27 Issue 46, page 33. Online at http://www.processor.com/editorial/article.asp?article=articles%2Fp2746%2F09p46%2F09p46.asp, retrieved 2007-09-10
* MITRE: Common Event Expression (CEE) Proposed Log Standard. Online at http://cee.mitre.org, retrieved 2010-03-03
* MITRE: Common Event Expression (CEE) Proposed Log Standard. Online at http://cee.mitre.org, retrieved 2010-03-03
* NIST 800-92: Guide to Security Log Management. Online at http://csrc.nist.gov/publications/nistpubs/800-92/SP800-92.pdf, retrieved 2010-03-03


==External links==
==External links==
* [http://www.infoworld.com/d/data-explosion/infoworld-review-meeting-the-network-security-and-compliance-challenge-658 InfoWorld review and comparison of commercial Log Management products]
* [http://www.infoworld.com/d/data-explosion/infoworld-review-meeting-the-network-security-and-compliance-challenge-658 InfoWorld review and comparison of commercial Log Management products]



{{DEFAULTSORT:Log Management And Intelligence}}
{{DEFAULTSORT:Log Management And Intelligence}}

Latest revision as of 15:59, 20 February 2024

Log management is the process for generating, transmitting, storing, accessing, and disposing of log data. A log data (or logs) is composed of entries (records), and each entry contains information related to a specific event that occur within an organization’s computing assets, including physical and virtual platforms, networks, services, and cloud environments.[1]

The process of log management generally breaks down into:[2]

  • Log collection - a process of capturing actual data from log files, application standard output stream (stdout), network socket and other sources.
  • Logs aggregation (centralization) - a process of putting all the log data together in a single place for the sake of further analysis or/and retention.
  • Log storage and retention - a process of handling large volumes of log data according to corporate or regulatory policies (compliance).
  • Log analysis - a process that helps operations and security team to handle system performance issues and security incidents

Übersicht

[edit]

The primary drivers for log management implementations are concerns about security,[3] system and network operations (such as system or network administration) and regulatory compliance. Logs are generated by nearly every computing device, and can often be directed to different locations both on a local file system or remote system.

Effectively analyzing large volumes of diverse logs can pose many challenges, such as:

  • Volume: log data can reach hundreds of gigabytes of data per day for a large organization. Simply collecting, centralizing and storing data at this volume can be challenging.
  • Normalization: logs are produced in multiple formats. The process of normalization is designed to provide a common output for analysis from diverse sources.
  • Velocity: The speed at which logs are produced from devices can make collection and aggregation difficult
  • Veracity: Log events may not be accurate. This is especially problematic for systems that perform detection, such as intrusion detection systems.

Users and potential users of log management may purchase complete commercial tools or build their own log-management and intelligence tools, assembling the functionality from various open-source components, or acquire (sub-)systems from commercial vendors. Log management is a complicated process and organizations often make mistakes while approaching it.[4]

Logging can produce technical information usable for the maintenance of applications or websites. It can serve:

  • to define whether a reported bug is actually a bug
  • to help analyze, reproduce and solve bugs
  • to help test new features in a development stage

Terminology

[edit]

Suggestions were made[by whom?] to change the definition of logging. This change would keep matters both purer and more easily maintainable:

  • Logging would then be defined as all instantly discardable data on the technical process of an application or website, as it represents and processes data and user input.
  • Auditing, then, would involve data that is not immediately discardable. In other words: data that is assembled in the auditing process, is stored persistently, is protected by authorization schemes and is, always, connected to some end-user functional requirement.

Deployment life-cycle

[edit]

One view[citation needed] of assessing the maturity of an organization in terms of the deployment of log-management tools might use[original research?] successive levels such as:

  1. in the initial stages, organizations use different log-analyzers for analyzing the logs in the devices on the security perimeter. They aim to identify the patterns of attack on the perimeter infrastructure of the organization.
  2. with the increased use of integrated computing, organizations mandate logs to identify the access and usage of confidential data within the security perimeter.
  3. at the next level of maturity, the log analyzer can track and monitor the performance and availability of systems at the level of the enterprise — especially of those information assets whose availability organizations regard as vital.
  4. organizations integrate the logs of various business applications into an enterprise log manager for a better value proposition.
  5. organizations merge the physical-access monitoring and the logical-access monitoring into a single view.

See also

[edit]

References

[edit]
  1. ^ NIST SP 800-92r1, Cybersecurity Log Management Planning Guide
  2. ^ Kent, Karen; Souppaya, Murugiah (September 2006). Guide to Computer Security Log Management (Report). NIST. doi:10.6028/NIST.SP.800-92. S2CID 221183642. NIST SP 800-92.
  3. ^ "Leveraging Log Data for Better Security". EventTracker SIEM, IT Security, Compliance, Log Management. Archived from the original on 28 December 2014. Retrieved 12 August 2015.
  4. ^ "Top 5 Log Mistakes - Second Edition". Docstoc.com. Retrieved 12 August 2015.
[edit]
Retrieved from "https://en.wikipedia.org/w/index.php?title=Log_management&oldid=1209157331"