Other Configuration Options for Explorer Service for Linux

The Explorer Service for Linux uses the settings defined in the json format configuration file named appsetting.$ASPNETCORE_ENVIRONMENT.json

ASPNETCORE_ENVIRONMENT is an environment variable, with default value Production.

Note: Changes made to the appsettings file take effect when the Explorer Service is started.

There are many other options available to configure the Explorer Service for Linux. The configurations options are typically not changed from the initially configured values under normal operation.

Configure the Explorer for Linux WebHost (or Kestrel) Configuration

Appsetting Field Path

Description

 
Kestrel:
Endpoints:
Http:Url

Defines the http endpoint used by the Explorer "Url":"http://*:5000".

5000 is the default http port

 
Kestrel:
Endpoints:
Https:Url

Defines the https endpoint used by the explorer "Url": "https://*:5001".

5001 is the default port

 
Kestrel:
Endpoints:
Https:
Certificate:Path

Defines the path to the server pfx certificate on the Linux Server

"Path": ""

If the path supplied is not the full path to the pfx file then the certificate path is relative to the installation folder.

If the Path is not specified in the file then explorer will look for the certificate in the path defined in the environment variable ASPNETCORE_Kestrel__Certificates__Default__Path

 
Kestrel:
Endpoints:
Https:
Certificate: 
Password

Defines the password for the server pfx certificate

"Password": ""

If the password is not specified in the file then the explorer will use password defined in the environment variable ASPNETCORE_Kestrel__Certificates__Default__Password

 
AllowedHosts

Allow this server to host the service "AllowedHosts": "*

Configure the Explorer for Linux Authentication Mode

Appsetting Field Path

Description

 
HttpServer:
AuthMode

Authmode to be used

"AuthMode": "ActiveDirectory"

OR

"AuthMode": "Ldap"

 
HttpServer:
UseBinding
HttpServer:
BindingSettings

The default values for the API Binding and BindingSettings:* properties

   “UseBinding” : "WSHttpBinding",

   "BindingSettings": {

      **

    }

These settings should not be modified.

Configure the Explorer for Linux DataStore location

By default the Explorer Service for Windows captures and stores data from connected Tidal Masters in the installation folder under subfolders

  • \data – contains data files from Tidal Masters via MasterLink

  • \derived – contains summary data files produced by Explorer Service

  • \log –contains logs from Explorer Service, admin tool and masterlink

Appsetting Field Path

Description

 
ExplorerCoreConfig:
DataStore

DataStore setting for Explorer on Linux “DataStore” : "%BASEDIRECTORY%"

Initially, the DataStore value is set to %BASEDIRECTORY%, that is the installation folder for the service. An empty value for DataStore also defaults to the installation folder for the service.

The Explorer Service will check the configured DataStore on startup. If any of these checks fail, then service will fail to start.

  • DataStore must be an existing folder that can be found by the Service.

  • Service can read, write and delete files in the DataStore folder

Note: The logging destination for the Explorer Service is controlled separately by the ExplorerLogConfig:path setting.

Configure the Explorer for Linux Logging

The Explorer for Linux uses the Serilog library that can be configured as shown below:

Appsetting Field Path

Description

 
ExplorerLogConfig: path

Path to Explorer Service logging location "path":

Syntax: "%BASEDIRECTORY%/log/explorer_.log".

The log file name will be appended by datetime information depending on the rollingInterval chosen.

Note: If the supplied path cannot be accessed then the Explorer Service will output an explorer log file into the log folder in the installation folder.

Note: The logging destination for the masterlink component is controlled separately by the ExplorerCoreConfig:DataStore on linux

 
ExplorerLogConfig: rollingInterval

Sets the period used to create new log files "rollingInterval": "Day".

  • Hour – append yyyyMMddHH to log filename

  • Day – append yyyyMMdd to log filename

  • Month – append yyyyMM to log filename

 
ExplorerLogConfig:retainedFileCountLimit

Set the upper limit on the number of log files retained "retainedFileCountLimit": 31.

Would retain 31 days of logs if the rollingInterval is Day

 
ExplorerLogConfig: restrictToMinimumLevel

Minimum Logging level applied "restrictToMinimumLevel": "Warning".

  • Debug

  • Verbose

  • Information

  • Warning

  • Fatal

 
ExplorerLogConfig: logRequests

Determines if web Requests are logged.

"logRequests": false

Should only be set to true for debugging purposes. Also requires restrictToMinimumLevel=Information

 
ExplorerLogConfig: logToConsole

Determines if log messages are send to the console.

"logToConsole": false

Should only be set to true for debugging purposes. These settings do not apply if the explorerserver is run via the exp.sh shell script.

Configuring Explorer Service for Linux Memory Limits for Tidal Master Data

The Explorer Service reads various data types from files, created by the MasterLink component, that correspond to different data sources from the Master such as master status, connections, system activity, and queues.

Example: Increasing the MasterStatisticsBuffer size increases the time span of data that can be displayed in the instance dashboard charts. 

Appsetting Fields
"ExplorerCoreConfig": {
  "MaxSystemActivityBuffer": 2000,
  "MaxConnectionsBuffer": 3600,
  "MaxConnectionDeletionBuffer": 1000,
  "MaxConnectionStateChangedBuffer": 2000,
  "MaxQueuesBuffer": 3600,
  "MaxStatisticsBuffer": 20000,
  "MaxMasterStatusBuffer": 3600,
  "MaxFMLogBuffer": 2000,
  "MaxFMLogStateChangeBuffer": 2000,
},

Configuring Explorer Service for Linux MasterLink Data Collection Settings

The MasterLink component collects information from the various configured Tidal instance masters and writes to various data files. The frequency of the DataCollection, DataWriteInterval, and DataFileSize can be configured. 

Appsetting Fields
"ExplorerCoreConfig": {
  "DataCollectInterval": 10,
  "DataWriteInterval": 60,
  "DataFileSize": 10,
},

The minimum allowed DataCollectionInterval is 10 seconds. Specifying a value lower than 10 seconds is ignored and a 10-second value is used by the MasterLink.

The minimum allowed DataWriteInterval is 60 seconds. Specifying a value lower than 60 seconds is ignored and a 60 second value is used by the MasterLink.

Configuring Explorer Service for Linux Cache Durations

The Explorer Service caches certain information for performance purposes.

Appsetting Field Path

Description

 
ExplorerCoreConfig: ClientLoginCacheExpMins

The duration, in minutes, that an Explorer Client’s user authentication status from MasterLink is cached before being re-authenticated by the service.

Default: 60 minutes

Maximum: 10 hours

This setting only applies when LDAP or ActiveDirectory Authentication is configured.

 
ExplorerCoreConfig:    
MasterUTCOffsetCacheExpMins

The duration, in minutes, that a Master’s MasterUTCOffset and MasterCurrentTime is cached by the service before being re-checked.

Default: 10 minutes

Minimum: 1 minute

 
ExplorerCoreConfig:    
MasterClientStatusCacheExpSecs

The duration, in seconds, that a Master’s (MasterLink Client) status is cached by the service before being re-checked.

Default: 15 seconds

Minimum: 5 seconds

Increasing this duration will delay updates to the Master and Instance Dashboards that relate to changes of state of a Tidal Master.

 
ExplorerCoreConfig:    
DerivedStatsFileCacheCount

The number of production days of derived statistics data that the service will cache in service memory. The cache is shared by all configured Tidal instances.

Default: 30

 
ExplorerCoreConfig:    
DerivedStatsFileCacheAbsExpirySecs

An absolute expiry timeframe, in seconds, for each individual cached production day of derived statistics data. A cached item is released after the absolute expiry period has elapsed.

Default: 1800 seconds (30 mins)

 
ExplorerCoreConfig:   
DerivedStatsFileCacheSlideExpirySecs

A sliding timeframe, in seconds, for each cached production day of derived statistics data. The expiry window for the cached entry is extended on each access to the entry - but not beyond the absolute expiry window for that entry.

Default: 60 seconds (1 min)

Configuring Explorer Service for Linux License Behavior

Explorer Clients acquire a license from the Explorer Service if a license slot is available when the Client initially connects to the service. Explorer Clients connect to the Explorer Service on a regular basis (every few seconds) to renew their license or acquire a license slot should one become available.

The Explorer Service tracks each Explorer Client session license. If an Explorer Client does not connect to the Explorer Service within a designated timeframe, that Client’s license is marked as expired. Expired licenses can be acquired by other clients. Expired licenses may be purged from the Explorer Service after an additional designated timeframe.

Appsetting Field Path

Default Value

Description

 
ExplorerCoreConfig:    
ClientLicenseExpiresAfter

00:00:30

The timespan since the last client connection to the service after which the client license is deemed to have expired.

 
ExplorerCoreConfig:    
ClientLicensePurgeAfter

00:02:00

The timespan since the last client connection to the service after which the client license is eligible to be purged from the Explorer Service.

Configure Explorer Service for Linux Output Search Result Retention

Job Output search result files produced by the Explorer Service can be retained for a configurable period.

Appsetting Field Path

Description

 
ExplorerCoreConfig:    
OutputSearchRetentionDays

The number of days output search results are to be stored by the Explorer Service.

Default: 31 day

Minimum: 1 day

Controlling the Explorer Service for Linux

Start the Explorer Service

Open a command prompt in the installation explorer folder and run the command ./exp.sh start

Example: /home/tidal/tidal/explorer/LINUX/tmkdea64 /home/tidal/tidal/explorer/explorerservice start

Check the Running Status of the Explorer Service

Open a command prompt in the installation explorer folder and run the command ./exp.sh status

Example: Process name:explorerservice pid:2006714 responding:True listening on port 5001

Note: Additional diagnostic information can be obtained for the status (and the stop) control options from the exp.sh script by replacing the line DEBUG=”false” with DEBUG=”true” in the Global Variable Definitions section of the script content.

Stop the Explorer Service

Open a command prompt in the installation explorer folder and run the command ./exp.sh stop

Example: /home/tidal/tidal/explorer/explorerservice stop

Get the version of the Explorer Service

Open a command prompt in the installation explorer folder and run the command ./exp.sh version

Example: TidalExplorerService 'Copyright © Tidal Software 2021' Version:3.1.0 BuiltType:Development Build:'3.1.0.2429' BuildDate:'20210708-190409' OS: RuntimeID:ubuntu.20.04-x64  API Versions Supported:10

Verifying Explorer Service Operation for Linux

Review the latest ExplorerService Log File for errors

Review the latest ExplorerService log file in logs\explorer_<timestamp> log file.

On successful start the latest log should contain a line of the form.

2021-07-16 11:31:33.957 -05:00 [WRN] Explorer Service App Starting

To get more verbose explorer logging information, set the logging level in the appsetting file to Information and restart the service.

Review the Latest Masterlink component Log files for errors

The masterlink-timestamp.out file contains the stdout and stderr from the MasterLink component that connects to the Tidal Masters.

Upon successful start, the masterlink would contain information of the form:

[03/30 15:12:07:484]:MasterLink: version 3.1.5.68
[03/30 15:12:07:484]:Java version: 17.0.6
[03/30 15:12:07:484]:Java Virtual Machine version: 17.0.6+10-LTS
[03/30 15:12:07:484]:Start Time : 03/30/23 15:12:07:484
[03/30 15:12:07:484]:----------------------------------------------------------------------------
[03/30 15:12:46:383]:Maximum number of log files = 50
[03/30 15:12:46:383]:Added a LogFile called 'tidalserver53'
[03/30 15:12:46:539]:ClientNode: Primary Server = [tcp://172.21.244.53:6215]
[03/30 15:12:46:539]:ClientNode: Backup Server  = [null]
[03/30 15:12:46:539]:Retrieved a LogFile called 'tidalserver53'
[03/30 15:12:46:586]:PrimaryServer=172.21.244.89:6215
[03/30 15:12:46:602]:DataCollectInterval=10
[03/30 15:12:46:602]:DataWriteInterval=60
[03/30 15:12:46:602]:DataFileSize=10
[03/30 15:12:47:216]:Retrieved a LogFile called 'tidalserver53'
[03/30 15:12:47:216]:Registered environment [tidalserver53] connects to master version [6.5.12.51]