Audits logs

Saagie uses audit logs to track user activity. This page will review the standard log outputs for Saagie’s main components, including the actions logged and how information appears in the output.

1. Projects and jobs

When a user changes an application’s resources, the following information is attached to the standard log output of the projects-and-jobs-api pod:

Resources include projects, jobs, pipelines, and apps, as well as their instances, plus Docker credentials and environment variables.
[AUDIT-{LOG_VERSION}] {DATEFORMAT_PATTERN_UTC} [{THREAD}] - {AUTHOR} {ACTION} {RESOURCE_TYPE} {RESOURCE_NAME} -[{LOG_METADATA}]-
  • {LOG_VERSION}: current log version

  • {DATEFORMAT_PATTERN_UTC}: all log times are in UTC

  • {THREAD}: thread name

  • {AUTHOR}: user performing the action

  • {ACTION}: action performed

  • {RESOURCE_TYPE}: resource type targeted by the action

  • {RESOURCE_NAME}: resource name targeted by the action

  • {LOG_METADATA}: describes metadata, such as realm, author, and action

If the action is UPDATE, UPGRADE, ROLLBACK, SET MAJOR VERSION, or UNSET MAJOR VERSION, a diff is added to the log:

old=<previous_value>, new=<updated_value>

If it is any other action, the resource’s current value appears in this format:

old=<current_value>, new=

The following actions are logged:

  • CREATE

  • UPDATE

  • DELETE

  • ARCHIVE

  • RUN

  • STOP

  • ROLLBACK

  • UPGRADE

  • SET MAJOR VERSION

  • UNSET MAJOR VERSION

Environment variables recovered without a project_id are global environment variables.

2. Technology manager

When a user changes a repository in the Technology Catalog, the following information is attached to the standard log output of the technology-manager pod:

[AUDIT-{LOG_VERSION}] {DATEFORMAT_PATTERN_UTC} [{THREAD}] - {AUTHOR} {ACTION} {RESOURCE_TYPE} {RESOURCE_NAME} -[{LOG_METADATA}]-
  • {LOG_VERSION}: current log version

  • {DATEFORMAT_PATTERN_UTC}: all log times are in UTC

  • {THREAD}: thread name

  • {AUTHOR}: user performing the action

  • {ACTION}: action performed

  • {RESOURCE_TYPE}: resource type targeted by the action

  • {RESOURCE_NAME}: resource name targeted by the action

  • {LOG_METADATA}: describes metadata, such as realm, author, and action

If the action is CREATE, the resource’s new value appears in this format:

old=, new=<new_value>

If the action is SYNCHRONIZE, the resource’s new value appears in this format:

old=, new=<updated_value>, previous_technologies=<previous_technologies>, updated_technologies=<updated_technologies>

If the action is UPDATE, a diff is added to the log:

old=<previous_value>, new=<updated_value>

If the action is DELETE, the resource’s current value appears in this format:

old=<current_value>, new=, technologies_removed=<technologies_removed>

The following actions are logged:

  • CREATE

  • UPDATE

  • DELETE

  • SYNCHRONIZE

3. Auth (users and groups)

When an administrator creates, updates, or deletes a group or a user, the following information is attached to the standard log output of the auth pod, inside of the auth container:

[AUDIT-{LOG_VERSION}] {DATEFORMAT_PATTERN_UTC} [{THREAD}] - {AUTHOR} {ACTION} {RESOURCE_TYPE} {RESOURCE_NAME} -[{LOG_METADATA}]-
  • {LOG_VERSION}: current log version

  • {DATEFORMAT_PATTERN_UTC}: all log times are in UTC

  • {THREAD}: thread name

  • {AUTHOR}: user performing the action

  • {ACTION}: action performed

  • {RESOURCE_TYPE}: resource type targeted by the action

  • {RESOURCE_NAME}: resource name targeted by the action

  • {LOG_METADATA}: describes metadata, such as realm, author, and action

The following actions are logged:

  • CREATE

  • UPDATE

  • DELETE

4. User authentications

When a user logs in to Saagie, changes their own password, or attempts to reset their password, the following information is attached to the standard log output of the authentication pod, inside of the authentication container:

[AUDIT-{LOG_VERSION}] {DATEFORMAT_PATTERN_UTC} [{THREAD}] - {AUTHOR} {ACTION} -[{LOG_METADATA}]-
  • {LOG_VERSION}: current log version

  • {DATEFORMAT_PATTERN_UTC}: all log times are in UTC

  • {THREAD}: thread name

  • {AUTHOR}: user performing the action

  • {ACTION}: action performed

  • {LOG_METADATA}: describes metadata, such as realm, author, and action

The following actions are logged:

  • CONNECT

  • FAILED_CONNECT

  • CHANGE_PASSWORD

  • TRIGGER_RESET_PASSWORD

  • RESET_PASSWORD

5. User profiles

When a user updates their email address or job title in their user profile, the following information is attached to the standard log output of the profile pod:

[AUDIT-{LOG_VERSION}] {DATEFORMAT_PATTERN_UTC} [{THREAD}] - {AUTHOR} {ACTION} {RESOURCE_TYPE} {RESOURCE_NAME} -[{LOG_METADATA}]-
  • {LOG_VERSION}: current log version

  • {DATEFORMAT_PATTERN_UTC}: all log times are in UTC

  • {THREAD}: thread name

  • {AUTHOR}: user performing the action

  • {ACTION}: action performed

  • {RESOURCE_TYPE}: resource type targeted by the action

  • {RESOURCE_NAME}: resource name targeted by the action

  • {LOG_METADATA}: describes metadata, such as realm, author, and action

The following action is logged:

  • UPDATE

When a user adds or updates their email address, the standard log output will show that an update was made, but it will not display the user’s email address.

6. Group authorizations

When a group’s authorizations are modified, the following information is attached to the standard log output of the security pod:

[AUDIT-{LOG_VERSION}] {DATEFORMAT_PATTERN_UTC} [{THREAD}] - {AUTHOR} {ACTION} -[{LOG_METADATA}]-
  • {LOG_VERSION}: current log version

  • {DATEFORMAT_PATTERN_UTC}: all log times are in UTC

  • {THREAD}: thread name

  • {AUTHOR}: user performing the action

  • {ACTION}: action performed

  • {LOG_METADATA}: describes metadata, such as realm, author, and action

The following actions are logged:

  • CREATE

  • UPDATE

  • DELETE

  • SET_IDENTIFIABLE_PERMISSION

  • REMOVE_IDENTIFIABLE_PERMISSION

    Setting and removing identifiable permissions is a specific type of group update. For example, when you add or remove a group’s permissions to view, edit, or manage a specific project with $PROJECT-NAME, you are setting an identifiable permission.

7. Parsing logs with Logstash

This pattern can be used for third-party applications to retrieve relevant information. Below is an example using Logstash.

\[%{WORD:log_type}-%{WORD:log_type_version}\] %{TIMESTAMP_ISO8601:timestamp} \[%{DATA:thread}\] - %{DATA:message} (?<![\\\\])-\[%{DATA:audit_logs_metadata}(?<![\\\\])\]-

We use a logstash plugin called kv to generate key/value pairs from the payload named audit_logs_metadata. Follow this pattern:

kv{
  source => "audit_logs_metadata"
  value_split => "="
  field_split => ","
  trim_key => " "
  include_keys => [ "realm", "platform_id", "author", "action", "project_id", "resource_name", "resource_type", "resource_id", "ip_address", "thread" ]
}