The Data Source Editor tool is introduced with inorigo 4.8. It enables users to define and save connections to- and configurations of data sources. These data sources can then be used throughout the inorigo suite and external tools via the API. The Data Source Editor provides essential capabilities for data integration in inorigo.

The sidebar provides an overview of the available Connections and Data Sources. Select an existing item to view details and to make changes. The Sidebar can be toggled by pressing the  pin.

Use the Search Field located at the top of the Sidebar to search with contains for any Data Source or Connection.

 

The  displays the type of Connection / Data Source

 

 

Used X time(s) displays the number of usages for a Connection / Data Source. Select an item and press Used X times in the top-right corner to see the usages.

If a Connection / Data Source is in use it cannot be deleted. To delete a Connection / Data Source in use – first find and delete or clear the usages.

 OK – The Connection / Data Source works is available.

 Pending – inorigo is waiting for the Connection / Data Source to respond.

A Connection / Read Timeout can be configured from the Connection settings.

 Fail – The Connection / Data Source could not be established. Hover the Fail icon to display additional information

Open in new tab	Opens Data Source/Connection in new tab
Open in Data Source/Connection Editor	Opens Data Source/Connection
Create copy	Opens a Copy of the selected Data Source or Connection with the given name Copy of [Unit Name]. Notice that the copy is not saved to the database until you press
Copy ID	Copies the ID of the Data Source/Connection
Bookmark Data Source/Connection	Adds Data Source/Connection to the Library

Create a Connection to set up a type and a location, such as an SQL server… or a Folder… Once set, Data Sources can make use of a connection. Create a Data Source to set up what content from a location that is to be included in the data source. Once the Data Source has been saved it will be available for use throughout the inorigo software.

You can use the same connection for all data sources that share a location and type. Say that you have an integration folder with multiple files that are to be used as Data Sources, using the same connection for all data sources is fast and easy to update if the integration folder is relocated. You can always change what connection a data source make use of later on. Here I have made use of the same connection for these two sources.

Simply press Create Connection in the left sidebar menu under the tab Connections to create a new connection. Once it has been created it will be available from the list of connections in the left sidebar.

Once a connection has been created it can be opened and edited by selecting it from the left sidebar menu. Make your changes and then press Save Changes to apply them. Leaving the page without pressing Save, will not apply any changes.

A connection in use cannot be deleted. To delete a connection first delete or redirect the data sources that use the connection. Click Usages: 

in the top-right corner displays where a connection is being used.

It’s possible to click the steps in the wizard to jump through steps when editing a Connection

 Create a connection to a Folder Path. Files located within the folder can be used when a data source is established.

 

 

Name (required)		Name the connection to find and use it in Data Sources and the Data Source Editor
Connection Type (required)		Choose the type of connection. Once a type is chosen, Type specific settings are revealed.
Folder (required)		Enter a path to the folder The default folder path is set to the inorigo integration folder: C:\Program Files \Apache Foundation\inorigoXYZ\inorigo\integration
User	Password	Enter a User name and Password to access the given folder path. For paths where the inorigo instance has access rights, no User or Password is required. Notice that its the inorigo-user running the server that requires access, not you the user setting up the Connection. By default, inorigo has access to the integration folder.
Test Connection		Test if the connection is working. Notice that test connection only tests the path, not user authentication for the data source.
Create Connection / Save Changes		Create the connection to save it to the Data Source Editor. Once a connection has been created it can be opened and edited at any time.

Name (required)		Name the connection to find and use it in Data Sources and the Data Source Editor
Connection Type (required)		Choose the type of connection. Once a type is chosen, Type specific settings are revealed.
Base URL (required)		Enter a Base URL
User	Password	Enter a User name and Password to access the given folder URL. For URLs where the inorigo instance has access rights, no User or Password is required. Notice that its the inorigo-user running the server that requires access, not you the user setting up the Connection. By default, inorigo has access to the integration folder.
Test Connection		Test if the connection is working. Notice that test connection only tests the path, not user authentication for the data source.
Create Connection / Save Changes		Create the connection to save it to the Data Source Editor. Once a connection has been created it can be opened and edited at any time.

 

 

Advanced

 

Connection Timeout (required)		Set a time (in seconds) to attempt to establish a connection. The connection will fail when if it has not been established when the given timeout has been exceeded. Default: 20 seconds
Read Timeout (required)		Set a timeout (in seconds) for a data source to be loaded from the connection. The connection will fail if the data sources has not been loaded when the given timeout has been exceeded. Default: 20 seconds
Request Headers: Key	Request Headers: Value	HTTP header fields are components of the header section of request and response messages. Use the request Key and Value fields to define a header field name and value to request. Example: Key: Content-Length Value: 348 More information about header fields can be found here

 

Name (required)		Name the connection to find and use it in Data Sources and the Data Source Editor
Connection Type (required)		Choose the type of connection. Once a type is chosen, Type specific settings are revealed.
Host (required)	Port	Enter the server address and if required a port. Example: 192.168.0.1:144 Default port is 1433
Integrated Security (required)		When yes inorigo will use the account that the service is running under to authorize the connection to the host. When no two input fields appear to provide a username and password.
User (required)	Password (required)	If integrated security is disabled, Enter a User name and Password to access the given SQL Server.  Test Connection Can be used to determine if access is granted.
Database (required)		Choose the database from the list of available databases from the server.
Test Connection		Test if the connection is working. The test button require login credentials to work for SQL types, unless  integrated security is used.
Create Connection / Save Changes		Create the connection to save it to the Data Source Editor. Once a connection has been created it can be opened and edited at any time.

 

Advanced

 

Instance		Enter the name of the SQL-instance if there are many instances.
Extra Parameters: Key	Extra Parameters: Value	Add extra parameters to the connection string.

 

Data Sources can make use of a connection. Create a Data Source to set up what content from a location that is to be saved as a Data Source. Once saved, a Data Source is available for use in inorigo.

Simply press Create Data Source in the left sidebar menu under the tab Data Sources to create a new Data Source.

A 4-step wizard appears where a Name and Data Source Type is first chosen. The choice of Data Source Type and Connection in step 1 and 2 of the wizard determine what settings that are available for step 3 and 4.

Information for each of the available Data Sources can be found here.

Information for each of the available types of Connections can be found here.

Once it has been created it will be available from the list of Data Sources in the left sidebar, in Knowledge Sets under External from the Data Type menu.

Once a Data Source has been created it can be opened and edited by selecting it from the left sidebar menu. Make your changes and then press Save Changes to apply them. Leaving the page without pressing Save, will not apply any changes.

A Data Source in use cannot be deleted. To delete a connection first delete or redirect the data sources that use the connection. Click Usages  in the top-right corner displays where a connection is being used.

It’s possible to click the steps in the wizard to jump through steps when editing a Data Source

1. Name & Type

Data Source Name (required)	Name the Data Source to find and use it in Data Source Editor and throughout inorigo.
Data Source Type (required)	Choose the type of Data Source to create. Once a type is chosen, type-specific settings are in the next step of the wizard.

 

 

2. Connection

Choose an available connection for the selected Data Source Type. If the desired connection has not yet been created, press Create New Connection.

Read more about the Connections types and settings here.

 

 

3.A – Based on a Folder Connection

Target (required)	Target either a single file by typing in Filename or target many files by typing in a Filename pattern (using Windows Wildcard symbols – more information here. When multiple files are targeted they are required to have the same Delimiter and Header row
Source Character Set	Set the character set that the source file is encoded in. Default character set is UTF-8
Delimiter (required)	Choose which symbol that separates the data in the CSV-file(s) into columns. ·         Comma (,) ·         Semicolon (;) ·         Space ( ) ·         Tab ( ) ·         Other Delimiter reveals a new field where a custom delimiter can be set.
Header Row	Enter the number of the row where the Header are placed in the files. The first row in the file is written as zero in the input field. If no headers are available in the file/files headers will be automatically be named as: #1, #2, #3, #4 Default: 0
First Data Row (required)	Enter the row number for the first data row to be included in the Data Source, so that inorigo will know on what row it should start reading the data. Default: 1 (Header is 0)
Last Data Row	Set a last row to include in the Data Source – rows below will not be read and will not be available in the data source. Only set a last data row if you do not want to include all rows below the first data row. Leaving the field blank will automatically include all rows (including rows that are continuously added to the source file) below the first data row.

 

 

3.B – Based on a URL Connection

 

Method (required)		Choose HTTP Request method. Chosen method works as a request-response protocol between the URL and inorigo. You can read more about HTTP Request methods here Typical: POST or GET Default: GET
SOAP Action		Specifies the SOAP (Simple Object Access Protocol) action to use in the HTTP header when invoking this method as a web method via HTTP
Accept (required)		The Accept request-header field is used to specify file types that are acceptable for the response. Typical: Application/xml Application/json Default: text/xml
Content-Type (required)		Default: text/xml; charset=utf-8
Body		Request body if Method is set to POST
Query  String Key	Query String Value	Enter additional parameters to the URL address to assign specific values. ??
Target (required)		Enter the path to the target file. Target is added to the path defined by the connection. The full given path is displayed underneath the Target field.
Source Character Set		Set the character set that the source file is encoded in. Defaults to: Windows 1251
Delimiter (required)		Choose which symbol that separates the data in the CSV-file(s) into columns. ·         Comma (,) ·         Semicolon (;) ·         Space ( ) ·         Tab ( ) Other Delimiter reveals a new field where a custom delimiter can be set.
Header Row		Enter the number of the row where the Header are placed in the files. The first row in the file is written as zero in the input field. If no headers are available in the file/files headers will be automatically be named as: #1, #2, #3, #4 Default: 0
First Data Row (required)		Enter the row number for the first data row to be included in the Data Source, so that inorigo will know on what row it should start reading the data. Default: 1 (Header is 0)
Last Data Row		Set a last row to include in the Data Source – rows below will not be read and will not be available in the data source. Only set a last data row if you do not want to include all rows below the first data row. Leaving the field blank will automatically include all rows (including rows that are continuously added to the source file) below the first data row.

 

4. Columns

Columns has the same settings for every type of data source, you can find them here

Applicable Formats

Tested and supported formats are: XLS, XLSX and XLXM. Although not tested, it is likely that other formats based on Apache POI such as Open Office also works.

1. Name & Type

Data Source Name (required)	Name the Data Source to find and use it in Data Source Editor and throughout inorigo.
Data Source Type (required)	Choose the type of Data Source to create. Once a type is chosen, type-specific settings are in the next step of the wizard.

 

2. Connection

Choose an available connection for the selected Data Source Type. If the desired connection has not yet been created, press Create New Connection.

Read more about the Connections types and settings here.

 

3.A – Based on a Folder Connection

Target (required)	Target either a single file by typing in Filename or target many files by typing in a Filename pattern (using Windows Wildcard symbols – more information here. The full path as it is defined in the Connection and the Target field is displayed under the entry field. When multiple files are targeted they are required to use the same sheet number, header row and first data row.
Source Character Set	Set the character set that the source file is encoded in. Defaults to Windows 1251
Sheet number (required)	·         Enter the Excel sheet number. It is not possible to read from more than one sheet per Data Source. ·         Default: 0
Header Row	Enter the number of the row where the Header are placed in the files. The first row in the file is written as zero in the input field. If no headers are available in the file/files headers will be automatically be named as: #1, #2, #3, #4 Default: 0
First Data Row (required)	Enter the row number for the first data row to be included in the Data Source, so that inorigo will know on what row it should start reading the data. Default: 1 (Header is 0)
Last Data Row	Set a last row to include in the Data Source – rows below will not be read and will not be available in the data source. Only set a last data row if you do not want to include all rows below the first data row. Leaving the field blank will automatically include all rows (including rows that are continuously added to the source file) below the first data row.

 

3.B – Based on a URL Connection

Method (required)		Choose HTTP Request method. Chosen method works as a request-response protocol between the URL and inorigo. You can read more about HTTP Request methods here Typical: POST or GET Default: GET
SOAP Action		Specifies the SOAP (Simple Object Access Protocol) action to use in the HTTP header when invoking this method as a web method via HTTP
Accept (required)		The Accept request-header field is used to specify file types that are acceptable for the response. Typical: Application/xml Application/json Default: text/xml
Content-Type (required)		Default: text/xml; charset=utf-8
Body		Request body if Method is set to POST
Query  String Key	Query String Value	Enter additional parameters to the URL address to assign specific values. ??
Target (required)		Enter the path to the target file. Target is added to the path defined by the connection. The full given path is displayed underneath the Target field.
Source Character Set		Set the character set that the source file is encoded in. Defaults to: Windows 1251
Sheet Number (required)		·         Enter the Excel sheet number. It is not possible to read from more than one sheet in per Data Source. Default: 0
Header Row		Enter the number of the row where the Header are placed in the files. The first row in the file is written as zero in the input field. If no headers are available in the file/files headers will be automatically be named as: #1, #2, #3, #4 Default: 0
First Data Row (required)		Enter the row number for the first data row to be included in the Data Source, so that inorigo will know on what row it should start reading the data. Default: 1 (Header is 0)
Last Data Row		Set a last row to include in the Data Source – rows below will not be read and will not be available in the data source. Only set a last data row if you do not want to include all rows below the first data row. Leaving the field blank will automatically include all rows (including rows that are continuously added to the source file) below the first data row.

 

4. Columns

Columns has the same settings for every type of data source, you can find them here

Format

HTML needs to be Well Formed, meaning that it must adhere to the syntax rules specified by the XML 1.0 specification in that it must satisfy both physical and logical structures. Read more about Well Formed documents here

1. Name & Type

Data Source Name (required)	Name the Data Source to find and use it in Data Source Editor and throughout inorigo.
Data Source Type (required)	Choose the type of Data Source to create. Once a type is chosen, type-specific settings are in the next step of the wizard.

 

 

2. Connection

Choose an available connection for the selected Data Source Type. If the desired connection has not yet been created, press Create New Connection.

Read more about the Connections types and settings here.

 

3.A – Based on a Folder Connection

Target (required)	Target either a single file by typing in Filename or target many files by typing in a Filename pattern (using Windows Wildcard symbols – more information here. The full path as it is defined in the Connection and the Target field is displayed under the entry field. When multiple files are targeted they are required to have the same Header Row and First Data row
Source Character Set	Set the character set that the source file is encoded in. Defaults to Windows 1251
Header Row	Enter the number of the row where the Header are placed in the files. The first row in the file is written as zero in the input field. If no headers are available in the file/files headers will be automatically be named as: #1, #2, #3, #4 Default: 0
First Data Row (required)	Enter the row number for the first data row to be included in the Data Source, so that inorigo will know on what row it should start reading the data. Default: 1 (Header is 0)
Last Data Row	Set a last row to include in the Data Source – rows below will not be read and will not be available in the data source. Only set a last data row if you do not want to include all rows below the first data row. Leaving the field blank will automatically include all rows (including rows that are continuously added to the source file) below the first data row.

 

3.B – Based on a URL Connection

Method (required)		Choose HTTP Request method. Chosen method works as a request-response protocol between the URL and inorigo. You can read more about HTTP Request methods here Typical: POST or GET Default: GET
SOAP Action		Specifies the SOAP (Simple Object Access Protocol) action to use in the HTTP header when invoking this method as a web method via HTTP
Accept (required)		The Accept request-header field is used to specify file types that are acceptable for the response. Typical: Application/xml Application/json Default: text/xml
Content-Type (required)		?? Default: text/xml; charset=utf-8
Body		Request body if Method is set to POST
Query  String Key	Query String Value	Enter additional parameters to the URL address to assign specific values. ??
Target (required)		Enter the path to the target file. Target is added to the path defined by the connection. The full given path is displayed underneath the Target field.
Source Character Set		Set the character set that the source file is encoded in. Defaults to: Windows 1251
Sheet Number (required)		·         Enter the Excel sheet number. It is not possible to read from more than one sheet in per Data Source. Default: 0
Header Row		Enter the number of the row where the Header are placed in the files. The first row in the file is written as zero in the input field. If no headers are available in the file/files headers will be automatically be named as: #1, #2, #3, #4 Default: 0
First Data Row (required)		Enter the row number for the first data row to be included in the Data Source, so that inorigo will know on what row it should start reading the data. Default: 1 (Header is 0)
Last Data Row		Set a last row to include in the Data Source – rows below will not be read and will not be available in the data source. Only set a last data row if you do not want to include all rows below the first data row. Leaving the field blank will automatically include all rows (including rows that are continuously added to the source file) below the first data row.

 

4. Columns

Columns has the same settings for every type of data source, you can find them here

1. Name & Type

Data Source Name (required)	Name the Data Source to find and use it in Data Source Editor and throughout inorigo.
Data Source Type (required)	Choose the type of Data Source to create. Once a type is chosen, type-specific settings are in the next step of the wizard.

 

2. Connection

Choose an available connection for the selected Data Source Type. If the desired connection has not yet been created, press Create New Connection.

Read more about the Connections types and settings here.

 

3. Data Source

Mode (required)

Table: If selected – all available tables will be presented in a drop down list.     Statement: If selected – A statement entry field will be displayed. Use it to enter an SQL statement to define what to include in the Data Source

 

4. Columns

Columns has the same settings for every type of data source, you can find them here

Format

XML needs to be Well Formed, meaning that it must adhere to the syntax rules specified by the XML 1.0 specification in that it must satisfy both physical and logical structures. Read more about Well Formed documents here

1. Name & Type

Data Source Name (required)	Name the Data Source to find and use it in Data Source Editor and throughout inorigo.
Data Source Type (required)	Choose the type of Data Source to create. Once a type is chosen, type-specific settings are in the next step of the wizard.

 

2. Connection

Choose an available connection for the selected Data Source Type. If the desired connection has not yet been created, press Create New Connection.

Read more about the Connections types and settings here.

 

3.A Based on a Folder Connection

Target (required)	Target either a single file by typing in Filename or target many files by typing in a Filename pattern (using Windows Wildcard symbols – more information here. The full path as it is defined in the Connection and the Target field is displayed under the entry field. When Multiple files are targeted in one data source they are required to follow the same configuration settings
Source Character Set	Set the character set that the source file is encoded in. Defaults to Windows 1251
Parse Mode	XPath uses path expressions to select nodes or node-sets in an XML document. Choose a Parse Mode to reveal additional entry fields.
Row Path (if Parse Mode = XPath) (required)	Enter a path in the following structure “/data/people/person”. Give a source for every column as a continuous search path starting for each row. If you set name as a column its possible to differentiate name as either an attribute for Person or its own attribute in Person: <data><People><Person><name>Alva</name></Person><Person name=”Amy“></Person></People></data>” Default: /TR
Row Tag (if Parse Mode = Table) (required)	Enter a tag that identifies a row in the XML sheet. Example: TR
Column Tag (required)	Enter a tag that identifies a column in the XML sheet. Example: TD
Table Path	Use to set what table to include if the XML contains more than one table. All tables are included if the entry field is left blank.
Header Row	Enter the number of the row where the Header are placed in the files. The first row in the file is written as zero in the input field. If no headers are available in the file/files headers will be automatically be named as: #1, #2, #3, #4 If the given Column tag is TD inorigo will automatically look for TH to find table headers. Default: 0
First Data Row (required)	Enter the row number for the first data row to be included in the Data Source, so that inorigo will know on what row it should start reading the data. Default: 1 (Header is 0)
Last Data Row	Set a last row to include in the Data Source – rows below will not be read and will not be available in the data source. Only set a last data row if you do not want to include all rows below the first data row. Leaving the field blank will automatically include all rows (including rows that are continuously added to the source file) below the first data row.

Advanced

XSLT	Enter XSLT to transform the structure of the XML document before its read by inorigo. This can be used, for example, to change the order of the content, to add or remove information, or selecting selected parts of the basic document.

 

 

3.B – Based on a URL Connection

Method (required) Add more		Choose http Request method. Chosen method works as a request-response protocol between the URL and inorigo. Default: GET
SOAP Action Add more		Specifies the SOAP (Simple Object Access Protocol) action to use in the HTTP header when invoking this method as a web method via HTTP
Accept (required) Add more		The Accept request-header field is used to specify file types that are acceptable for the response. Default: text/xml
Content-Type (required)		Charset for the content Default: text/xml; charset=utf-8
Body		Request body if Method is set to POST
Query  String Key	Query String Value	Enter additional parameters to the URL address to assign specific values. ??
Target (required)		Enter the path to the target file. Target is added to the path defined by the connection. The full given path is displayed underneath the Target field.
Source Character Set		Set the character set that the source file is encoded in. Defaults to: Windows 1251
Parse Mode		XPath uses path expressions to select nodes or node-sets in an XML document. Choose a Parse Mode to reveal additional entry fields.
Row Path (if Parse Mode = XPath) (required)		Enter a path in the following structure “/data/people/person”. Give a source for every column as a continuous search path starting for each row. If you set name as a column its possible to differentiate name as either an attribute for Person or its own attribute in Person: <data><People><Person><name>Alva</name></Person><Person name=”Amy“></Person></People></data>” Default: /TR
Row Tag (if Parse Mode = Table) (required)		Enter a tag that identifies a row in the XML sheet. Example: TR
Column Tag (required)		Enter a tag that identifies a column in the XML sheet. Example: TD
Table Path		Use to set what table to include if the XML contains more than one table. All tables are included if the entry field is left blank.
Header Row		Enter the number of the row where the Header are placed in the files. The first row in the file is written as zero in the input field. If no headers are available in the file/files headers will be automatically be named as: #1, #2, #3, #4 Notice! If the given Column tag is TD inorigo will automatically look for TH to find table headers. Default: 0
First Data Row (required)		Enter the row number for the first data row to be included in the Data Source, so that inorigo will know on what row it should start reading the data. Default: 1 (Header is 0)
Last Data Row		Set a last row to include in the Data Source – rows below will not be read and will not be available in the data source. Only set a last data row if you do not want to include all rows below the first data row. Leaving the field blank will automatically include all rows (including rows that are continuously added to the source file) below the first data row.

 

Advanced

XSLT	Enter XSLT to transform the structure of the XML document before its read by inorigo. This can be used, for example, to change the order of the content, to add or remove information, or selecting selected parts of the basic document.

 

4. Columns

Columns has the same settings for every type of data source, you can find them here

 

Columns

Columns allows you to

1. Choose what columns to include
2. Format Names

Press Add Columns to reveal all columns.

If the columns do not load as intended at this stage, make sure that the connection and the Data Source is properly configured.

The Add Columns Pop-up displays all columns with the Name, Data Type and order given in the source.

You can make changes to how inorigo interpret and display these columns in the Data Source – but can always come back to the Add Columns Pop-up to view the columns as they are defined in the source.

Columns as they are defined by the source

Tick the checkboxes of all columns that are to be included in the Data Source.

Add columns to create duplicates or to define different Data Types for one column. Calculated columns are not available in the Data Source Editor, import a data source to a knowledge set to use this functionality.

The columns FILE_TIME and FILE_NAME are included for CSV, Excel, XML type Data Sources

FILE_TIME	Includes date and time of when the file was created or last modified
FILE_NAME	Includes the full path to the files location on the server

 

Changes in data type will cause inorigo to interpret the columns as the given Data Type.

Click Add Columns to include the ticked columns in the Data Source.

 

The included columns appear in the Columns field as the image displayed below.

Names and Data Type can be modified.

Notice that these changes are saved in inorigo and will not be applied to the source. To view the columns as they are defined by the source, press Add Columns.

 

It’s possible to add manual columns below the list of available columns. Use the Source ID given in the right column to refer to an available column.

 

Filter Data Source

The query tool can be used to apply custom filters to the Data Source.

Press Define query to bring up the query tool – you can learn more about how to write queries here

Tick distinct rows to remove all duplicate rows.

 

Preview Data Source

Preview displays a preview of the first rows in the data source.

Use preview to verify that applied filters and changes to the columns works as intended.

Once finish is pressed, the Data Source is saved and will be available to use throughout the inorigo suite.

If a Data Source/Connection has not yet been saved, name it and press Create. If changes are made to an existing Data Source/Connection, Save is available.

	Create a Data Source/Connection
	Apply changes (overwrites previous save)
	Delete Data Source/Connection
	Unable  to delete Data Source/Connection. Either because it has not yet been created, or because the Data Source/Connection is  in use. See Usages