Audit Logs

Saagie uses audit logs to track user activity. You must know the standard log outputs for the main Saagie components, including the actions logged and how information appears in the output.

Projects and Jobs

When you change an app resources, the following information is attached to the standard log output of the projects-and-jobs-api pod:

Resources include projects, jobs, pipelines, apps, and their instances, plus Docker credentials and environment variables.

[AUDIT-{LOG_VERSION}] {DATEFORMAT_PATTERN_UTC} [{THREAD}] - {AUTHOR} {ACTION} {RESOURCE_TYPE} {RESOURCE_NAME} -[{LOG_METADATA}]-

Where:

{LOG_VERSION} is the current version of the log.
{DATEFORMAT_PATTERN_UTC} is the time of the log. As a reminder, all log times are in UTC.
{THREAD} is the thread name.
{AUTHOR} is the user executing the action.

{ACTION} is the executed action.

The possible values are CREATE, UPDATE, DELETE, RESTART, RUN, STOP, ROLLBACK, UPGRADE, SET MAJOR VERSION, and UNSET MAJOR VERSION.

If the action is UPDATE, UPGRADE, ROLLBACK, SET MAJOR VERSION, or UNSET MAJOR VERSION, a diff value is added to the log as follows:
```
old=<previous_value>, new=<updated_value>
```
If it is any other action, the current value of the resource appears as follows:
```
old=<current_value>, new=
```

Note that the environment variables that are retrieved without a project_id are global environment variables.

{RESOURCE_TYPE} is the resource type targeted by the action.
{RESOURCE_NAME} is the resource name targeted by the action.
{LOG_METADATA} describes the metadata, such as realm, author, and action.

Technology Manager

When a user changes a repository in the Technology Catalog, the following information is attached to the standard log output of the technology-manager pod:

[AUDIT-{LOG_VERSION}] {DATEFORMAT_PATTERN_UTC} [{THREAD}] - {AUTHOR} {ACTION} {RESOURCE_TYPE} {RESOURCE_NAME} -[{LOG_METADATA}]-

Where:

{LOG_VERSION} is the current version of the log.
{DATEFORMAT_PATTERN_UTC} is the time of the log. As a reminder, all log times are in UTC.
{THREAD} is the thread name.
{AUTHOR} is the user executing the action.

{ACTION} is the executed action.

The possible values are CREATE, SYNCHRONIZE, UPDATE, and DELETE.

CREATE
SYNCHRONIZE
UPDATE
DELETE

If the action is CREATE, the new value of the resource appears as follows:

old=, new=<new_value>

If the action is SYNCHRONIZE, the new value of the resource appears as follows:

old=, new=<updated_value>, previous_technologies=<previous_technologies>, updated_technologies=<updated_technologies>

If the action is UPDATE, a diff is added to the log as follows:

old=<previous_value>, new=<updated_value>

If the action is DELETE, the current value of the resource appears as follows:

old=<current_value>, new=, technologies_removed=<technologies_removed>

{RESOURCE_TYPE} is the resource type targeted by the action.
{RESOURCE_NAME} is the resource name targeted by the action.
{LOG_METADATA} describes the metadata, such as realm, author, and action.

Users and Groups

When an administrator creates, updates, or deletes a group or a user, the following information is attached to the standard log output of the auth pod:

[AUDIT-{LOG_VERSION}] {DATEFORMAT_PATTERN_UTC} [{THREAD}] - {AUTHOR} {ACTION} {RESOURCE_TYPE} {RESOURCE_NAME} -[{LOG_METADATA}]-

Where:

{LOG_VERSION} is the current version of the log.
{DATEFORMAT_PATTERN_UTC} is the time of the log. As a reminder, all log times are in UTC.
{THREAD} is the thread name.
{AUTHOR} is the user executing the action.
{ACTION} is the executed action.

The possible values are CREATE, UPDATE, and DELETE.
{RESOURCE_TYPE} is the resource type targeted by the action.
{RESOURCE_NAME} is the resource name targeted by the action.
{LOG_METADATA} describes the metadata, such as realm, author, and action.

User Authentications

When a user logs into Saagie, changes their own password, or attempts to reset it, the following information is attached to the standard log output of the authentication pod:

[AUDIT-{LOG_VERSION}] {DATEFORMAT_PATTERN_UTC} [{THREAD}] - {AUTHOR} {ACTION} -[{LOG_METADATA}]-

Where:

{LOG_VERSION} is the current version of the log.
{DATEFORMAT_PATTERN_UTC} is the time of the log. As a reminder, all log times are in UTC.
{THREAD} is the thread name.
{AUTHOR} is the user executing the action.
{ACTION} is the executed action.

The possible values are CONNECT, FAILED_CONNECT, CHANGE_PASSWORD, TRIGGER_RESET_PASSWORD, and RESET_PASSWORD.
{LOG_METADATA} describes the metadata, such as realm, author, and action.

User Profiles

When a user changes their email address or job title in their user profile, the following information is attached to the standard log output of the profile pod:

When a user adds or updates their email address, the standard log output indicates that an update was made, but it does not display the user’s email address.

[AUDIT-{LOG_VERSION}] {DATEFORMAT_PATTERN_UTC} [{THREAD}] - {AUTHOR} {ACTION} {RESOURCE_TYPE} {RESOURCE_NAME} -[{LOG_METADATA}]-

Where:

{LOG_VERSION} is the current version of the log.
{DATEFORMAT_PATTERN_UTC} is the time of the log. As a reminder, all log times are in UTC.
{THREAD} is the thread name.
{AUTHOR} is the user executing the action.
{ACTION} is the executed action.

The possible value is UPDATE.
{RESOURCE_TYPE} is the resource type targeted by the action.
{RESOURCE_NAME} is the resource name targeted by the action.
{LOG_METADATA} describes the metadata, such as realm, author, and action.

Group Authorizations

When a group authorizations are modified, the following information is attached to the standard log output of the security pod:

[AUDIT-{LOG_VERSION}] {DATEFORMAT_PATTERN_UTC} [{THREAD}] - {AUTHOR} {ACTION} -[{LOG_METADATA}]-

Where:

{LOG_METADATA}: describes metadata, such as realm, author, and action
{LOG_VERSION} is the current version of the log.
{DATEFORMAT_PATTERN_UTC} is the time of the log. As a reminder, all log times are in UTC.
{THREAD} is the thread name.
{AUTHOR} is the user executing the action.
{ACTION} is the executed action.

The possible values are CREATE, UPDATE, DELETE, SET_IDENTIFIABLE_PERMISSION, and REMOVE_IDENTIFIABLE_PERMISSION.
{LOG_METADATA} describes the metadata, such as realm, author, and action.

Setting and removing identifiable permissions is a specific type of group update. For example, when you add or remove group permissions to view, edit, or manage a specific project with $PROJECT-NAME, you are setting an identifiable permission.

Parsing Logs with Logstash

The following pattern can be used by third party applications to retrieve relevant information:

\[%{WORD:log_type}-%{WORD:log_type_version}\] %{TIMESTAMP_ISO8601:timestamp} \[%{DATA:thread}\] - %{DATA:message} (?<![\\\\])-\[%{DATA:audit_logs_metadata}(?<![\\\\])\]-

We use a Logstash plugin called kv to generate key and value pairs from the payload named audit_logs_metadata. It has the following pattern:

kv{
  source => "audit_logs_metadata"
  value_split => "="
  field_split => ","
  trim_key => " "
  include_keys => [ "realm", "platform_id", "author", "action", "project_id", "resource_name", "resource_type", "resource_id", "ip_address", "thread" ]
}