xref: /openbmc/docs/designs/phosphor-audit.md (revision f4febd00)

# phosphor-audit

Author: Ivan Mikhaylov, [i.mikhaylov@yadro.com](mailto:i.mikhaylov@yadro.com)

Other contributors: Alexander Amelkin,
[a.amelkin@yadro.com](mailto:a.amelkin@yadro.com) Alexander Filippov,
[a.filippov@yadro.com](mailto:a.filippov@yadro.com)

Created: 2019-07-23

## Problem Description

End users of OpenBMC may take actions that change the system state and/or
configuration. Such actions may be taken using any of the numerous interfaces
provided by OpenBMC. That includes RedFish, IPMI, ssh or serial console shell,
and other interfaces, including the future ones.

Consequences of those actions may sometimes be harmful and an investigation may
be conducted in order to find out the person responsible for the unwelcome
changes. Currently, most changes leave no trace in OpenBMC logs, which hampers
the aforementioned investigation.

It is required to develop a mechanism that would allow for tracking such user
activity, logging it, and taking certain actions if necessary.

## Background and References

YADRO had an internal solution for the problem. It was only applicable to an
outdated version of OpenBMC and needed a redesign. There was also a parallel
effort by IBM that can be found here:
[REST and Redfish Traffic Logging](https://gerrit.openbmc.org/c/openbmc/bmcweb/+/22699)

## Assumptions

This design assumes that an end user is never given a direct access to the
system shell. The shell allows for direct manipulation of user database
(add/remove users, change passwords) and system configuration (network scripts,
etc.), and it doesn't seem feasible to track such user actions taken within the
shell. This design assumes that all user interaction with OpenBMC is limited to
controlled interfaces served by other Phosphor OpenBMC components interacting
via D-Bus.

## Requirements

- Provide a unified method of logging user actions independent of the user
  interface, where possible user actions are:
  - Redfish/REST PUT/POST/DELETE/PATCH
  - IPMI
  - PAM
  - PLDM
  - Any other suitable service
- Provide a way to configure system response actions taken upon certain user
  actions, where possible response actions are:
  - Log an event
  - Notify an administrator or an arbitrary notification receiver
  - Run an arbitrary command
- Provide a way to configure notification receivers:
  - E-mail
  - SNMP
  - Instant messengers
  - D-Bus

## Proposed Design

The main idea is to catch D-Bus requests sent by user interfaces, then handle
the request according to the configuration. In future, support for flexible
policies may be implemented that would allow for better flexibility in handling
and tracking.

The phosphor-audit service represents a service that provides user activity
tracking and corresponding action taking in response of user actions.

The key benefit of using phosphor-audit is that all action handling will be kept
inside this project instead of spreading it across multiple dedicated interface
services with a risk of missing a handler for some action in one of them and
bloating the codebase.

The component diagram below shows the example of service overview.

```ascii
  +----------------+  audit event                           +-----------------+
  |    IPMI NET    +-----------+                            | action          |
  +----------------+           |                            | +-------------+ |
                               |                            | |   logging   | |
  +----------------+           |                            | +-------------+ |
  |   IPMI HOST    +-----------+      +--------------+      |                 |
  +----------------+           |      |    audit     |      | +-------------+ |
                               +----->+   service    +----->| |   command   | |
  +----------------+           |      |              |      | +-------------+ |
  |  RedFish/REST  +-----------+      +--------------+      |                 |
  +----------------+           |                            | +-------------+ |
                               |                            | |   notify    | |
  +----------------+           |                            | +-------------+ |
  |  any service   +-----------+                            |                 |
  +----------------+                                        | +-------------+ |
                                                            | |     ...     | |
                                                            | +-------------+ |
                                                            +-----------------+
```

The audit event from diagram generated by an application to track user activity.
The application sends 'signal' to audit service via D-Bus. What is happening
next in audit service's handler depends on user requirements and needs. It is
possible to just store logs, run arbitrary command or notify someone in handler
or we can do all of the above and all of this can be optional.

**Audit event call**

Audit event call performs preprocessing of incoming data at application side
before sending it to the audit service, if the request is filtered out, it will
be dropped at this moment and will no longer be processed. After the filter
check, the audit event call sends the data through D-Bus to the audit service
which makes a decision regarding next steps. Also, it caches list of possible
commands (blacklist or whitelist) and status of its service (disabled or
enabled). If the service in undefined state, the call checks if service alive or
not.

> `audit_event(type, rc, request, user, host, data)`
>
> - type - type of event source : IPMI, REST, PAM, etc.
> - rc - return code of the handler event (status, rc, etc.)
> - request - a generalized identifier of the event, e.g. ipmi command
>   (cmd/netfn/lun), web path, or anything else that can describe the event.
> - user - the user account on behalf of which the event was processed. depends
>   on context, NA/None in case of user inaccessibility.
> - source - identifier of the host that the event has originated from. This can
>   be literally "host" for events originating from the local host (via locally
>   connected IPMI), or an IP address or a hostname of a remote host.
> - data - any supplementary data that can help better identify the event (e.g.,
>   some first bytes of the IPMI command data).

Service itself can control flow of events with configuration on its side.

Pseudocode for example:

    audit_event(NET_IPMI, "access denied"(rc=-1), "ipmi cmd", "qwerty223",
                          "192.168.0.1", <some additional data if needed>)
    audit_event(REST, "login successful"(rc=200), "rest login",
                      "qwerty223", "192.168.0.1", NULL)
    audit_event(HOST_IPMI, "shutting down the host"(rc=0), "host poweroff",
                       NULL, NULL, NULL)

`audit_event(blob_data)` Blob can be described as structure:

    struct blob_audit
    {
        uint8_t type;
        int32_t rc;
        uint32_t request_id;
        char *user;
        sockaddr_in6 *addr;
        struct iovec *data;
    }

When the call reaches the server destination via D-Bus, the server already knows
that the call should be processed via predefined list of actions which are set
in the server configuration.

Step by step execution of call:

- client's layer
  1. checks if audit is enabled for such service
  2. checks if audit event should be whitelisted or blacklisted at the audit
     service side for preventing spamming of unneeded events to audit service
  3. send the data to the audit service via D-Bus
- server's layer
  1. accept D-Bus request
  2. goes through list of actions for each services

How the checks will be processed at client's layer:

1.  check the status of service and cache that value
2.  check the list of possible actions which should be logged and cache them
    also
3.  listen on 'propertiesChanged' event in case of changing list or status of
    service

## Service configuration

The configuration structure can be described as tree with set of options, as
example of structure:

```
[IPMI]
   [Enabled]
   [Whitelist]
     [Cmd 0x01] ["reset request"]
     [Cmd 0x02] ["hello world"]
     [Cmd 0x03] ["goodbye cruel world"]
   [Actions]
     [Notify type1] [Recipient]
     [Notify type2] [Recipient]
     [Notify type3] [Recipient]
     [Logging type] [Options]
     [Exec] [ExternalCommand]
[REST]
   [Disabled]
   [Blacklist]
     [Path1] [Options]
     [Path2] [Options]
   [Actions]
     [Notify type2] [Recipient]
     [Logging type] [Options]
```

Options can be updated via D-Bus properties. The audit service listens changes
on configuration file and emit 'PropertiesChanged' signal with changed details.

- The whitelisting and blacklisting

> Possible list of requests which have to be filtered and processed. 'Whitelist'
> filters possible requests which can be processed. 'Blacklist' blocks only
> exact requests.

- Enable/disable the event processing for directed services, where the directed
  service is any suitable services which can use audit service.

> Each audit processing type can be disabled or enabled at runtime via config
> file or D-Bus property.

- Notification setup via SNMP/E-mail/Instant messengers/D-Bus

> The end recipient notification system with different transports.

- Logging

> phosphor-logging, journald or anything else suitable for.

- User actions

> Running a command as consequenced action.

## Workflow

An example of possible flow:

```ascii
           +----------------+
           |   NET   IPMI   |
           |    REQUEST     |
           +----------------+
                   |
 +--------------------------------------------------------------------------+
 |         +-------v--------+                                         IPMI  |
 |         |    NET IPMI    |                                               |
 |         +----------------+                                               |
 |                 |                                                        |
 |         +-------v--------+        +---------------------------+          |
 |         | rc = handle()  +------->|  audit_event<NET_IPMI>()  |          |
 |         +----------------+        +---------------------------+          |
 |                 |                              |                         |
 |                 |                              |                         |
 |         +-------v--------+                     |                         |
 |         |   Processing   |                     |                         |
 |         |    further     |                     |                         |
 |         +----------------+                     |                         |
 +--------------------------------------------------------------------------+
                                                  |
                                                  |
 +--------------------------------------------------------------------------+
 |                  +-----------------------------+                         |
 |                  |                                        Audit Service  |
 |                  |                                                       |
 |                  |                                                       |
 |                  |                                                       |
 |            +-----v------+                                                |
 |        NO  | Is logging |        YES                                     |
 |     +------+  enabled   +--------------------+                           |
 |     |      | for  type? |                    |                           |
 |     |      +------------+            +-------v-----+                     |
 |     |                           NO   | Is request  |   YES               |
 |     |                       +--------+    type     +--------+            |
 |     |                       |        |  filtered?  |        |            |
 |     |                       |        +-------------+        |            |
 |     |                       |                               |            |
 |     |               +-------v-------+                       |            |
 |     |               |    Notify     |                       |            |
 |     |               | Administrator |                       |            |
 |     |               +---------------+                       |            |
 |     |                       |                               |            |
 |     |               +-------v-------+                       |            |
 |     |               |   Log Event   |                       |            |
 |     |               +---------------+                       |            |
 |     |                       |                               |            |
 |     |               +-------v-------+                       |            |
 |     |               |     User      |                       |            |
 |     |               |    actions    |                       |            |
 |     |               +---------------+                       |            |
 |     |                       |                               |            |
 |     |               +-------v-------+                       |            |
 |     +-------------->|      End      |<----------------------+            |
 |                     +---------------+                                    |
 |                                                                          |
 +--------------------------------------------------------------------------+
```

## Notification mechanisms

The unified model for reporting accidents to the end user, where the transport
can be:

- E-mail

  > Sending a note to directed recipient which set in configuration via sendmail
  > or anything else.

- SNMP

  > Sending a notification via SNMP trap messages to directed recipient which
  > set in configuration.

- Instant messengers

  > Sending a notification to directed recipient which set in configuration via
  > jabber/sametime/gtalk/etc.

- D-Bus

  > Notify the other service which set in configuration via 'method_call' or
  > 'signal'.

Notifications will be skipped in case if there is no any of above configuration
rules is set inside configuration. It is possible to pick up rules at runtime.

## User Actions

- Exec application via 'system' call.
- The code for directed handling type inside handler itself. As example for 'net
  ipmi' in case of unsuccessful user login inside handler:
  - Sends a notification to administrator.
  - echo heartbeat > /sys/class/leds/alarm_red/trigger

## Alternatives Considered

Processing user requests in each dedicated interface service and logging them
separately for each of the interfaces. Scattered handling looks like an
error-prone and rigid approach.

## Impacts

Improves system manageability and security.

Impacts when phosphor-audit is not enabled:

- Many services will have slightly larger code size and longer CPU path length
  due to invocations of audit_event().
- Increased D-Bus traffic.

Impacts when phosphor-audit is enabled: All of the above, plus:

- Additional BMC processor time needed to handle audit events.
- Additional BMC flash storage needed to store logged events.
- Additional outbound network traffic to notify users.
- Additional space for notification libraries.

## Testing

`dbus-send` as command-line tool for generating audit events.

Scenarios:

- For each supported service (such as Redfish, net IPMI, host IPMI, PLDM),
  create audit events, and validate they get logged.
- Ensure message-type and request-type filtering works as expected.
- Ensure basic notification actions work as expected (log, command, notify).
- When continuously generating audit-events, change the phosphor-audit service's
  configuration, and validate no audit events are lost, and the new
  configuration takes effect.