代码拉取完成，页面将自动刷新
This article is a quick entry document of Exchangis 1.0, covering the basic usage process of Exchangis 1.0. Exchangis a lightweight data exchange service platform, which supports data synchronization between different types of data sources. The platform data exchange process is split, and the concepts of data source, data exchange task and task scheduling are abstracted, so as to achieve the purpose of visual management of data synchronization process. In the actual data transmission process, the characteristics of multiple transmission components can be integrated to achieve horizontal expansion of functions.
Exchangis1.0 is currently a part of DSSdata exchange component, and it can be accessed in the component list by logging in to DSS. Therefore, before using Exchangis 1.0, please make basic deployment of DSS, Exchange IS 1.0, Linkis and other related components to ensure that the components' functions are available. This article will not go into details. See for details:exchangis_deploy_en和exchangis-appconn_deploy_en
By default, the system uses Linux deployment users of Linkis to log in to DSS. If users of hadoop deploy Linkis and DSS, they can log in directly through the account password: hadoop/hadoop. First log in to the webpage according to the front-end deployment address of DSS, and then enter the account password: hadoop/hadoop to enter DSS.
Exchangis is accessed through DSS. Click on the word: Home ->DSS Application Components-> Data Exchange-> Enter Exchange.
This module can configure and manage data sources. As the initial step of data synchronization, Exchangis1.0 currently supports direct data import between mysql and hive. The main functions of data source are as follows ：
Click Create Data Source and select the data source you want to create. Currently, MySQL and Hive data sources can be created.
Select the MySQL data source and fill in the configuration parameters, among which the asterisk is required. Make sure that the Host, port number, user name and password connected to MySQL database are connected correctly. Connection parameter is in Json format, which is used to set the configuration information of MySQL. After filling it out, you can test the connection.
For the configuration of Hive data source, it is different from MySQL. For the time being, it does not provide users with the function of configuring cluster parameters in the interface. For the cluster environment, it is completed by the back-end unified configuration. Users only need to select the required cluster environment and click OK to save it.
The data source management module provides the function of publishing the version of the configured data source. Only the published data source can be used when configuring derivative tasks, otherwise, it will prompt that it is unavailable. As long as the data source is edited again, it will be regarded as a new version, and the latest version is in the first row. You can view the configuration of all historical data source versions in the version list, and you can refer to it whenever you need to roll back.
The **expired ** function of data source management is used to indicate that this data source has been gradually replaced. Please change the task configuration using this data source in time to avoid the failure of the configured execution task caused by directly deleting the data source.
This module can create projects. In actual derivative tasks, there can be multiple derivative tasks under a project, and different projects do not affect each other. For ordinary users, they can only operate their own created projects. On the homepage of project management, you can manage projects, including create, modify and delete and query and search. Modification and deletion can only be performed on projects created in Exchangis.
Enter the project, and you can see the task list under the project.
You can manage the created Job data synchronization tasks in the task list, similar to projects, including Create, Modify, Delete and Search.
Click Create Task to select the task type and execution engine. Currently, only offline tasks and SQOOP execution engine are supported, and streaming tasks and DataX engines will be supported in the future.
Click Add subtask in the task to add multiple subtasks. You can modify, copy and delete subtasks.
At the same time, tasks support copying, and copied subtasks contain all information of their atomic task configuration.
Before this step, you need to add data sources in the data source management module for selection and publish. The current Exchangis version only supports **MySQL data sources and Hive data sources **.
Data synchronization task configuration and execution is the core function of Exchangis1.0.0. The basic configuration data synchronization process is: Add subtask-> Select Source data source and Sink data source-> Field mapping configuration-> Process control-> Task configuration-> Save-> Execute.
The main functions of task execution include:
For the newly created data synchronization subtask, you should select the data source library table, and the selected data source is the published data source in the data source module. Select the data source to support search, and the search method is to search the library first, and then search the table.
When MySQL is the destination data source, it supports insert and update two writing methods; When it is the source data source, it supports WHERE conditional statement query.
When Hive is the destination data source, partition information configuration is supported, and the writing methods are append data and overwrite; When the source data source is partition information configuration is supported.
When the data Source library table information is configured, Exchangis1.0.0 will automatically map the fields of source data source and Sink data source, and you can choose the fields you want to map by yourself, or check whether our fields match. When Hive is a Sink data source, its mapping field cannot be modified.
Task execution provides configuration of maximum number of parallel jobs (default is 1) and configuration of maximum job memory (default is 1024Mb), which can be changed according to actual needs.
Exchangis1.0 support the simultaneous execution of multiple subtasks. After the task configuration is completed, click Execute to start the data synchronization task, and the workbench will pop up at the bottom of the interface. The workbench mainly contains three functions: running status, real-time log and execution history . Running Status :You can view the overall progress of the current data synchronization task, including the number of successes and failures of the task, and click on the name of the task to display the information of various running indicators of each task.
Real-time log :The main display contents include two categories. One is the log of the whole derivative job, which can output the status log of each task, such as whether the task is scheduled or not, and whether it is running moderately; The second is the log of each task, and the output is the corresponding derivative log. In the real-time log, the log can be filtered according to keywords and ignored words, and the function of obtaining the last N lines of logs is provided separately; You can also screen and display different types of logs of Error, Warning and Info, just click the corresponding button.
Execution History :It can display the historical execution information of this derivative task and provide a preliminary overview of the historical execution process. If you want to further view the detailed historical information, click on the task name to jump to the synchronous history interface for viewing.
Data synchronization task execution needs to specify the execution user, which is the login user by default. The specific situation needs to be adjusted according to the configuration of the actual data source.
This module can view all data synchronization tasks performed historically, and each user can only view the tasks created by himself. Different users are isolated from each other.
The main functions are as follows:
At present, Exchangis1.0 supports docking with DSS in the form of Appconn, on the DSS side, data exchange sqoop workflow node can be created in the mode of workflow arrangement through **application development-> project list ** of DSS, where data synchronization tasks can be configured and executed. Exchangis projects and data exchange tasks created in DSS will be created in Exchangis at the same time.
Exchangis Appconn mainly supports the following functions ：
**Project operation ** is the creation, deletion and modification of DSS projects, which will synchronously affect the projects on Exchangis；
**Basic operation of workflow node **is the task of creating, deleting and modifying sqoop workflow nodes in DSS choreographer, which will be synchronized to Exchangis.
It is the task of creating, deleting and modifying sqoop workflow nodes in DSS choreographer, which will be synchronized to Exchangis；
**Workflow derivative operation ** support sqoop workflow node configuration to perform data synchronization tasks；
**Workflow publishing operation ** support sqoop workflow nodes to publish to WTSS for task scheduling.
This module can create, modify and delete DSS projects, and the operations on the DSS side will be synchronized to the Exchange is side. Here, take the project created in DSS as an example, and the process of operation in Exchange is as follows: Click Create Project-> Fill in project information-> Click Confirm-> Enter Exchange is side-> Click Project Management, and you can view the synchronously created projects, as shown in the following figure:
After the creation, you will see the synchronized project on the Exchange is side.
Workflow nodes can be created, modified, deleted, and selected depending on each other, and can be associated with each other. Operations on DSS side will be synchronized to Exchangis side. Taking the creation of sqoop workflow node as an example, the operation process of the project in exchangis appconn is as follows: Create a workflow-> Drag sqoop node from the left plug-in bar to the right canvas-> Click OK to create sqoop node task-> Enter Exchangis to view the synchronously created task, as shown in the following figure, and the same is true for deleting and modifying sqoop node tasks.
You can see that the derivative task is also synchronized to Exchangis. 
Double-click the Sqoop node to operate the workflow node. sqoop workflow node configuration and data synchronization tasks are supported. Derivative task in the form of workflow nodes is the core function of Exchangis Appconn Each sqoop node represents a data synchronization task, and the specific operation process is as follows: Double-click the sqoop node-> Task configuration interface pops up-> Configure task information-> Execute task, as shown in the following figure：
There are two execution methods here, one is to click the execute button in the pop-up task configuration interface to execute; The other is to click the Execute button of DSS choreographer or select the Execute button to execute. Click Execute to execute all the nodes in this workflow, and click Select Execute to execute only the selected workflow nodes, but not all the nodes.
Note: For data synchronization tasks performed in sqoop node of DSS, relevant information can be viewed on Exchangis.
The publish function of workflow tasks supports sqoop workflow nodes to publish to WTSS for task scheduling. The data exchange task information created and configured in the * * Development Center can be published to WTSS, and the task can be scheduled in WTSS.
Click the drop-down box in the namespace and switch to **Production Center **, where you can see the workflow logs of all projects and check the scheduling status of each workflow.
此处可能存在不合适展示的内容，页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容，可点击提交进行申诉，我们将尽快为您处理。
