Log Management

1. Applications

   Applications log their activity in correlation with the business processes they support, particularly those operations that modify permissions or access rights.

   These logs generally include:

   • The business operation that was requested.
   • Whether the request was accepted or denied.
   • The time and date the operation was performed (Start and end times may be appropriate for long operations).
   • Who initiated the operation.
   • System and network resources used.
   • Any information needed for business process controls.
   • Client hardware and software characteristics.

2. Systems

   Many components of the IT infrastructure generate logs. Examples of these components include:

   • Operating Systems.
   • Web servers.
   • Database servers.
   • Print servers.
   • File servers.
   • Authentication servers.
   • DHCP servers.
   • DNS servers.
   • E-mail server logs.

3. Network devices

   Many components of the network infrastructure generate logs. Examples of these components include:

   • Routers.
   • Switches.
   • Wireless access points.
   • Network-based firewalls, intrusion detection and prevention systems, next-generation firewalls.

   These logs typically describe flows of information through the network, but not the individual packets contained in that flow. Information logged for a flow should include:

   • Network (IP) addresses or telephone numbers of the end points.
   • Service identifiers (port numbers) for each of the end points.
   • Whether the flow was accepted or denied.
   • Date, time, and duration of the flow.
   • Number of packets and bytes used by the flow.

4. Microsoft Azure

   Three types of platform logs are generated which record the following types of actions:

   • Azure Active Directory Reports – detailed changes made in Azure AD and login activity.
   • Activity logs – record operations performed on an Azure resource – such as creating a VM.
   • Resource logs – capture operations performed within an Azure resource, such as querying a database or writing to a storage bucket.

   Both Activity Logs and Resource Logs use the Log Analytics workspace to store and access these logs. Also, Log Analytics data is used/analyzed by SIEM to generate alerts/incidents based on defined analytics rules.

1. Operating System logs

   Operating systems (OS) for servers, workstations, and networking devices (e.g., routers, switches) usually log a variety of information related to security. FintechOS is logging the most common types of security-related OS data, as follows:

   • System Events - System events are operational actions performed by OS components, such as shutting down the system or starting a service. Each event is timestamped and other supporting information may include event, status, and error codes; service name; and user or system account associated with an event.
   • Audit Records - Audit records hold security event information such as successful and failed authentication attempts, file accesses, security policy changes, account changes (e.g. account creation and deletion, account privilege assignment), and use of privileges.

2. Applications/platform logs

   Logging level can take one of the following values: Verbose, Debug, Information, Warning, Error, Fatal. A minimum level of logging for each logging sink can be set.

   FintechOS is actively using two logging sinks which can be enabled together or separately: file logging and Application Insights logging. We also have SEQ logging already added to the platform and can add more Serilog sinks in future FintechOS releases (based on request).

   Type of logs that we expect to meet:

   • Platform

     • Exceptions in the execution of the FTOS platform logic.
     • Warning/Info messages added by the platform developers to have a better view on what happened in a certain workflow.
     • Errors where the exceptions are handled, and information is provided about the cause on why a specific logic failed.
     • Digital developers can decide to add their own logging with the desired log level in the automation scripts written for client specific implementations.
     • Items related to observability on the application performance/statistics are not handled at the platform level and should be extracted using, for example, AppInsights out-of-the-box capabilities.
     • Trace.log files configured by default with periodical switching to new files of logs. Trace log files are grabbed by external sources. (SIEM, log server).

   • Database

     There are four types of logging events which can be stored in the DB:

     • FintechOS Platform API Logging

       • Optional setting – based on customer requirements

       • Can be filtered by source type (i.e. OpenApi, Data Service) and method name

       • Logs are stored in EbsLogs.ApiLog

       • Attention needs to be paid to filtering and cleaning the logging table, as the high number of requests can increase the DB size considerably

       • Starting with 22.1.0 this can also be sent directly to the same logging sink as platform logging (no filtering on security, it is all or nothing)

     • CRUD Operations Logging

       • Optional setting – based on customer requirements

       • It can only be turned on and off without any other granularity settings – logging the CRUD events on all entities

       • Logs stored in EbsLogs.UniversalLog

       • Starting with 22.1.0 this can also be sent directly to the same logging sink as platform logging (no filtering on security, it is all or nothing)

     • Authentication Events logging

       • Starting with 24.1, go to the Identity Provider to enable logging user login data.

       • Data is stored in the Event_Entity table in the IDP database.

     • Data Audit

       Optional setting which by default is turned off

     NOTE  
     FintechOS is logging all CRUD and API operations executed in the platform, by default, in a separate database schema named EbsLogs. Database administrators can restrict read access for this schema and grant insert rights only for the SQL login used by the FintechOS platform. Granular rights can be set up.

3. DCS/DCI (Digital Cloud Services and Digital Cloud Infrastructure)

   Logs generated by integration with FTOS technology partners hold data needed for solution debugging, sometimes files sent to the partners and, in some cases, initial requests and responses. Logs are stored on virtual machines where solutions reside. Restricted access to these virtual machines is enforced.

In this example, we are anonymizing multiple customer emails and first names using a script that contains an array with 2 objects. In this scenario, Log anonymization is already enabled.

1. Edit the sys-log-anonymization-settings and add email, firstName in the Parameter Value field.

2. In Studio, create a new server script with an array with 2 objects:

   Copy

   example1 = new Object();
   example.email = "example1@mail.com";
   example.firstName = "Name";
   
   example2 = new Object();
   example.email = "example2@mail.com";
   example.firstName = "First";
   
   log([example1, example2]);

3. Create an endpoint for the script.

4. Open the console from the browser and execute the script: ebs.callActionByName("example_script").

5. Check the logs in App Insights and notice that the elements are anonymized.

You can anonymize multiple customer emails and first names using a script that contains 2 elements in Json:

1. Edit the sys-log-anonymization-settings and add email, firstName in the Parameter Value field.

2. In Studio, create a new server script that has 2 elements in Json:

   Copy

   log([{"email":"example1@mail.com", "firstName" = "Name"}, {"email":"example2@mail.com", "firstName" = "First"}]))

3. Create an endpoint for the script.

4. Open the console from the browser and execute the script: ebs.callActionByName("example_script",(res)=>console.log(res)).

5. Check the logs in App Insights and notice that the elements are anonymized.

In the following example we're filtering by two automation scripts, test_script_1 and test_script_2, and one digital asset, named test_da.

1. In Configuration Manager, go to Studio > App Settings.

2. Edit sdk-log-filters to add sdk-log-filters: {"automationscript":["test_script_1", "test_script_2"], "digital-asset":["test_da"]}.

3. Open Studio and run the following commands from the Console, one-by-one:

   Copy

   ebs.callActionByName('test_script_1')
   ebs.callActionByName('test_script_2')
   ebs.callActionByName('test_da')

4. Check the logs from App Insights. The scripts and the digital asset are displayed in the logs.