How to import your own data to JetOctopus

You can manually upload different data to JetOctopus if you need to include it in your crawl results or log analysis.

You can manually import the following file types:

• CSV;
• TXT;
• log files (you can use the default Apache and Nginx logs format).

By the way, when configuring crawling, you can add links to sitemaps or scan only sitemaps if you need audit sitemaps. You can also integrate data from Google Search Console and Ahrefs. If you need to manually upload this data, you can contact our technical support: we will definitely find a way!

How to import a dataset

For import, you can use a file in CSV or TXT format. During the upload process, you will be able to adjust the delimiter, charset, etc.

Once you’ve created the file you need to import, go to the needed project. Next, select the menu “Tools” – “My data” – “Import File”.

Drop CSV or TXT file in the special field or click on this field to upload the file.

Then configure the dataset:

• come up with an understandable name (this name will be displayed in the list of datasets in all sections);
• if the first row of the table is column names, activate the “First Row Is Header” checkbox; if your columns do not have names and the first line contains the required URL, then this checkbox should be deactivated;
• click on “Advanced settings” to configure delimiter, charset, enclosure and escape for your table; however, JetOctopus will automatically select the option if you do not know these settings;
• click the “Update” button and proceed to the next stage.

In the “Columns” block, configure the type of each column:

• URL Key, if there are URL addresses in the column;
• Integer – an integer (whole number) without text;
• String – can contain any text and any symbols;
• Float – floating numbers;
• Boolean – yes or no column.

In the “Data sample” you will see an example of an imported table. If everything is displayed correctly, click “Load data”.

If the data is not displayed as you want, change the settings in the “Advanced settings” field. For example, if you know that your file has three columns, but the “Columns” block shows one, change the delimiter to the desired one.

In the image below, we can see that the table had three columns: URL, status code 1, status code two. But we set the delimiter “,”. Therefore, all columns are displayed as one, because in our imported table the delimiter is “|” (it appears as part of the column). Therefore, change the delimiter and click the “Update” button to see how the imported table will look.

Done! You can save this dataset as a segment or combine this dataset with data from logs, crawls or Google Search Console.

At any time, you can view or change the dataset settings. To do this, go to “Tools” – “My data” – “Overview”. Next, select the desired dataset and edit or delete it.

How you can use manually uploaded data

We recommend using dataset upload in the following cases:

• if you need to scan a list of specific URLs, and the number of these URLs is greater than 100,000 lines (we can scan fewer URLs using the URL list mode);
• attach pages for data analysis from Google Search Console, logs, crawl (this will be needed if you have orphaned pages or a specific sample of pages for analysis).

All files that have been uploaded will be displayed in the dataset list in each section. That is, you can join datasets and merge them with any other data.

You can upload log files through the menu “Logs” – “Manage logs” – “Upload files”.