Quickstart¶
This section provides a demonstration of the capabilities of FactoryTX. You will configure the three major components of FactoryTX to initiate data acquisition from two different data sources. You will be using the FactoryTX UI for all the configuration requirements. You will also need access to a Sight Machine cloud environment if you want to see raw data transmitted to the platform (ask your Sight Machine contact for details). The guide uses sample data from an imaginary manufacturing startup that specializes in safari park vehicles. Given the nature of modern auto manufacturing, the supply chain is distributed all over the globe and raw data is stored in the cloud. The data is stored using two services, each with data related to different manufacturing processes. FactoryTX will be configured to collect data from an MS SQL Server and an Azure Data Lake, apply some light data transformations, and upload data as records to a Sight Machine cloud instance.
Steps that we will walk through:
Getting an API key¶
FactoryTX and the Sight Machine Manufacturing Analytics platform (MA) use secure authentication to verify the identity of the raw data source. In order to initiate the data upload process, an Edge API key will need to be generated and saved to the FactoryTX configuration.
Edge API keys are created and managed in MA. To view and manage them, navigate to your MA web UI in a browser.
Select AI Data Pipeline from the MA menu
Select Edge and then API keys from the horizontal toolbar
A list of Edge API keys will be displayed in the table. To create a new key, select the + icon in the upper right corner of the UI.
Running FactoryTX¶
There are a few ways to run FactoryTX:
locally with
docker-compose;on an appliance (IPC) managed by Sight Machine;
on a FactoryTX Virtual Machine (VM) managed by Sight Machine;
in Kubernetes (i.e. FactoryTX Cloud)
If you don’t have access to a working FactoryTX installation, please refer to the guide on setting up Local FactoryTX with Docker Compose.
Otherwise, please navigate to the FactoryTX Configuration Editor.
Configuring FactoryTX¶
In the Configuration Editor, replace the existing configuration with the following configuration:
1{
2 "sight_machine_cloud": {
3 "base_url": "<Please input your tenant URL>",
4 "api_key_id": "<Please input your Edge API key ID>",
5 "api_key": "<Please input your Edge API key>"
6 },
7 "data_receiver": [
8 {
9 "data_receiver_name": "SQL Plugin 1",
10 "protocol": "sql",
11 "poll_interval": 10,
12 "connections": [
13 {
14 "database_type": "mssql",
15 "host": "35.233.132.166",
16 "port": 1433,
17 "username": "rexy",
18 "password": "islaNublar93",
19 "database_name": "Demo",
20 "ssl": false,
21 "connection_name": "Nedry"
22 }
23 ],
24 "streams": [
25 {
26 "asset": "lasercut",
27 "stream_type": "mssql",
28 "state_fields": [
29 {
30 "column_name": "timestamp",
31 "data_type": "timestamp",
32 "initial_value": "2018-10-22 10:29:00"
33 }
34 ],
35 "query": "SELECT * FROM ca_lasercut\nWHERE timestamp > :timestamp\nORDER BY timestamp;"
36 },
37 {
38 "asset": "diecast",
39 "stream_type": "mssql",
40 "state_fields": [
41 {
42 "column_name": "TIMESTAMP",
43 "data_type": "timestamp",
44 "initial_value": "2018-10-22 12:29:00"
45 }
46 ],
47 "query": "SELECT * FROM hk_diecast\nWHERE TIMESTAMP > :TIMESTAMP\nORDER BY TIMESTAMP;"
48 },
49 {
50 "asset": "fusion",
51 "stream_type": "mssql",
52 "state_fields": [
53 {
54 "column_name": "timestamp",
55 "data_type": "timestamp",
56 "initial_value": "2018-10-22 02:28:00"
57 }
58 ],
59 "query": "SELECT * FROM tl_fusion WHERE timestamp > :timestamp ORDER BY timestamp;"
60 }
61 ]
62 },
63 {
64 "data_receiver_name": "azure_receiver_1",
65 "protocol": "azure_data_lake",
66 "poll_interval": 5,
67 "connections": [
68 {
69 "tenant_id": "beb1d7f9-8e2e-4dc4-83be-190ebceb70ea",
70 "client_id": "b8d454e3-67ec-4f2f-8dcb-a5d6f87abbc6",
71 "client_secret": "BdgGBpZ7pS.Ogf..7PpT0JVLXes.i1uh",
72 "store_name": "factorytxreceivertest",
73 "data_directory": "directory-1",
74 "max_files_per_request": 1,
75 "connection_name": "Blue"
76 }
77 ],
78 "parsers": [
79 {
80 "parser_name": "CSV_Parser_1",
81 "parser_type": "csv"
82 }
83 ],
84 "streams": [
85 {
86 "asset": "spraypaint",
87 "stream_type": "azure_lake",
88 "file_filter": [
89 "*.csv"
90 ],
91 "parser": "CSV_Parser_1"
92 }
93 ]
94 }
95 ],
96 "transforms": [
97 {
98 "transform_name": "Rename 1",
99 "transform_type": "rename",
100 "filter_stream": [
101 "diecast:mssql"
102 ],
103 "renames": [
104 {
105 "from": "TIMESTAMP",
106 "to": "timestamp"
107 }
108 ]
109 },
110 {
111 "transform_name": "Rename 2",
112 "transform_type": "rename",
113 "filter_stream": [
114 "spraypaint:azure_lake"
115 ],
116 "renames": [
117 {
118 "from": "Timestamp",
119 "to": "timestamp"
120 }
121 ]
122 },
123 {
124 "transform_name": "Time Converter 1",
125 "transform_type": "convert_timestamps",
126 "filter_stream": [
127 "spraypaint:*"
128 ],
129 "field_names": [
130 "timestamp"
131 ],
132 "timezone": "Asia/Tokyo"
133 },
134 {
135 "transform_name": "Time Converter 2",
136 "transform_type": "convert_timestamps",
137 "filter_stream": [
138 "lasercut:mssql"
139 ],
140 "field_names": [
141 "timestamp"
142 ],
143 "timezone": "America/New_York"
144 },
145 {
146 "transform_name": "Time Converter 3",
147 "transform_type": "convert_timestamps",
148 "filter_stream": [
149 "diecast:mssql"
150 ],
151 "field_names": [
152 "timestamp"
153 ],
154 "timezone": "Asia/Hong_Kong"
155 },
156 {
157 "transform_name": "Time Converter 4",
158 "transform_type": "convert_timestamps",
159 "filter_stream": [
160 "fusion:mssql"
161 ],
162 "field_names": [
163 "timestamp"
164 ],
165 "timezone": "Europe/Paris"
166 }
167 ],
168 "data_transmit": [
169 {
170 "transmit_name": "MyTransmit_1",
171 "transmit_type": "remotedatapost"
172 }
173 ]
174}
IMPORTANT: You’ll need to input the base_url, api_key_id, and
api_key values in the sight_machine_cloud section with values that
you generated in your MA service environment. Please refer to the
Getting an API key section for reference on how to create API keys.
In the exemplified configuration file, FactoryTX is collecting data from our international factories from two remote data sources: an MS SQL Server and an Azure Data Lake instance.
The MS SQL Server has three tables in the Demo database that we want:
“ca_lasercut”: The table contains information from 6 laser cutting machines at our factory in Carmel, Indiana. These machines create molds for our car parts.
“hk_diecast”: The table contains information from 3 die casting machines at our factory in Hong Kong, China. These machines pour molten compounds into the molds created by the laser cutting machines.
“tl_fusion”: The table contains information from our 1 fusion machine at our factory in Toulouse, France. The fusion machine assembles the separate parts into the whole car.
The Azure Data Lake instance has CSV files of data from our machines in Tokyo, Japan, which apply a few coats of spray paint to the exterior of our cars.
Before transmitting the data to the Sight Machine cloud, FactoryTX will apply some light data transformations, such as renaming a SQL table column and converting timestamps into the factory’s time zone.
To get more details about the configuration file and what each of its sections mean, please refer to the Receivers and Transforms and Transmits article.
Click the submit button in the FactoryTX Controller to save the configuration and FactoryTX will automatically begin collecting, transforming, and transmitting data to the Sight Machine cloud.
Verifying data has been transmitted¶
To verify that our manufacturing data has been successfully collected, transformed, and transmitted to the Sight Machine cloud, you’ll need to navigate to the AI Data Pipeline in your MA Service Environment.
Select Raw Data and then Raw Data Table from the horizontal toolbar
The Raw Data Table displays records in the Sight Machine cloud, organized based on assets and stream types of the records. To confirm that FactoryTX has transmitted all of the data, please check that there are 4 assets with data available: diecast, fusion, lasercut, and spraypaint.
The spraypaint asset should have 4 records (or rows in the table).
The diecast asset should have 3591 records for 1 stream type: mssql.
The fusion asset should have 1586 records for 1 stream type.
The lasercut asset should have 323 records for 1 stream type.
