OnCrawl is a powerful SEO crawler and log analyzer for enterprise SEO audits and daily monitoring. With over 400 customers in Europe, including the half of the TOP 20 French e-commerce websites, and in North America, OnCrawl helps e-commerce websites, SEO agencies and publishers improve their SEO performance, rankings and revenues.
OnCrawl also combines your logs and crawl data to provide comprehensive analytics about the on-page SEO factors that have a positive or negative impact on your indexation and crawl budget. OnCrawl gives you detailed data and analysis you can act on.
The team aims to democratize its tools, thanks to technological innovations, to let small but also bigger companies get access to the SEO data they need at affordable prices.
Oncrawl and AT Internet have developed a bidirectional bridge that allows you to:
Once you are here, you will need to access to your AT Internet interface in order to create the two API requests (see section below)
As soon as your two flows are created on AT Internet interface, you will have a GUID code:
Copy this GUID code and paste it on Oncrawl interface. Do not forget to write down you ID Site:
You are now able to launch the crawl. As soon as the crawl is finalized, you will see your crawled SEO data crossed with your AT Internet data.
API List
The list of Oncrawl API are listed below.
Oncrawl crossed with AT
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((oql:(field:!(ati_entrances_seo,gte,'1'))))&fmt=row_objects
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((oql:(and:!((field:!(depth,has_no_value,'')),(field:!(ati_entrances_seo,gte,'1'))))))&fmt=row_objects
- SEO visits from orphan pages
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((oql:(and:!((field:!(depth,has_no_value,'')),(field:!(ati_entrances_seo,gte,'1')))),value:'ati_entrances_seo:sum'))&fmt=row_objects
- SEO active pages by depth
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:depth)),oql:(and:!((field:!(ati_entrances_seo,gte,'1')),(field:!(depth,has_value,''))))))&fmt=row_objects
- Average inrank by SEO visits range
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:ati_entrances_seo,ranges:!((from:1,name:one,to:2),(from:2,name:'2_9',to:10),(from:10,name:'10_99',to:100),(from:100,name:'100_999',to:1000),(from:1000,name:'1000_9999',to:10000),(from:10000,name:'10000_99999',to:100000),(from:100000,name:more_100000)))),oql:(and:!((field:!(ati_entrances_seo,gte,'1')),(field:!(depth,has_value,'')))),value:'inrank:avg'))&fmt=row_objects
- Load time evaluation by SEO visits range
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:ati_entrances_seo,ranges:!((from:1,name:one,to:2),(from:2,name:'2_9',to:10),(from:10,name:'10_99',to:100),(from:100,name:'100_999',to:1000),(from:1000,name:'1000_9999',to:10000),(from:10000,name:'10000_99999',to:100000),(from:100000,name:more_100000))),(name:load_time_range)),oql:(and:!((field:!(ati_entrances_seo,gte,'1')),(field:!(depth,has_value,''))))))&fmt=row_objects
- Average SEO bounce rate by load time evaluation
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:load_time_range)),oql:(and:!((field:!(ati_entrances_seo,gte,'1')),(field:!(depth,has_value,'')))),value:'ati_bounce_rate_seo:avg'))&fmt=row_objects
Structure
- Pages crawled: Pages discovered and fetched by OnCrawl during the crawl. Oncrawl bot considers robots.txt and no follow rules.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((oql:(field:!(fetched,equals,true))))&fmt=row_objects
- Indexable pages: Pages fetched by OnCrawl bot without noindex in meta robots. The content has been analyzed unlike pages whose meta robots is noindex.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((oql:(and:!((field:!(fetched,equals,true)),(field:!(meta_robots_index,equals,true)),(field:!(status_code_range,equals,ok))))))&fmt=row_objects
- Indexable compliant pages: Indexable HTML pages that returned a status code 200 to Oncrawl robot. Non compliant pages can be redirections, errors (status code 3xx or 404) or pages that are not HTML (xml, pdf ...)
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((oql:(and:!((field:!(meta_robots_index,equals,true)),(field:!(status_code,equals,'200')),(field:!(canonical_evaluation,not_equals,not_matching)),(field:!(parsed_html,equals,true))))))&fmt=row_objects
- Depth distribution: Shows how pages are distributed by depth. Page depth refers to the number of clicks you need to reach a specific page from the homepage using the shortest path. Depth matters insofar search engines are less likely to find deepest pages and so to crawl them
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:depth)),oql:(and:!((field:!(meta_robots_index,equals,true)),(field:!(status_code,equals,'200')),(field:!(canonical_evaluation,not_equals,not_matching)),(field:!(parsed_html,equals,true))))))&fmt=row_objects
Links breakdown
- Internal links follow: Links from pages of the site, pointing to other pages of the site. Follow attribute informs bots to crawl the link’s target and thus distribute any page rank to the destination page.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((oql:(and:!((field:!(depth,has_value,'')),(field:!(internal_follow_outlinks,gt,'0')))),value:'internal_follow_outlinks:sum'))&fmt=row_objects
- Internal links nofollow: Links from pages of the site, pointing to other pages of the site. Nofollow attribute informs bots to not crawl the link’s target and thus do not distribute any page rank to the destination page
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((oql:(and:!((field:!(depth,has_value,'')),(field:!(internal_nofollow_outlinks,gt,'0')))),value:'internal_nofollow_outlinks:sum'))&fmt=row_objects
- External links follow: Links from pages of the site pointing to pages of another site. Follow attribute informs bots to crawl the link’s target and thus distribute any page rank to the destination page.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((oql:(and:!((field:!(depth,has_value,'')),(field:!(external_follow_outlinks,gt,'0')))),value:'external_follow_outlinks:sum'))&fmt=row_objects
- External links nofollow: Links from pages of the site pointing to pages of another site. Nofollow attribute informs bots to not crawl the link’s target and thus do not distribute any page rank to the destination page.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((oql:(and:!((field:!(depth,has_value,'')),(field:!(external_nofollow_outlinks,gt,'0')))),value:'external_nofollow_outlinks:sum'))&fmt=row_objects
- Inrank by depth: The inrank is a kind of internal pagerank. It is a score (on a 0 to 10 scale) developed by Oncrawl that helps to understand how popular pages are regarding the internal link architecture. Shows the impact of the depth on the internal pagerank
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:depth)),oql:(and:!((field:!(meta_robots_index,equals,true)),(field:!(status_code,equals,'200')),(field:!(canonical_evaluation,not_equals,not_matching)),(field:!(parsed_html,equals,true)))),value:'inrank:avg'))&fmt=row_objects
Main tags
- Title evaluation: Provides a clear view of your titles state of play. Titles tags are the main tags on which you should focus. They have a real impact on your search engine results and your page ranking, so it is necessary to fill them correctly.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:title_evaluation)),oql:(and:!((field:!(meta_robots_index,equals,true)),(field:!(status_code,equals,'200')),(field:!(canonical_evaluation,not_equals,not_matching)),(field:!(parsed_html,equals,true))))))&fmt=row_objects
- Description evaluation: Provides a clear view of your main meta descriptions state of play. Even if Google has said that keywords in your meta description are not affecting your rankings, it can compel visitors to click on your site through the SERP if the keywords are the ones they were looking for.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:description_evaluation)),oql:(and:!((field:!(meta_robots_index,equals,true)),(field:!(status_code,equals,'200')),(field:!(canonical_evaluation,not_equals,not_matching)),(field:!(parsed_html,equals,true))))))&fmt=row_objects
- H1 evaluation: Provides a clear view of your titles state of play. H1 tags are as important as titles. They have a real impact on your search engine results and your page ranking, so it is necessary to fill them correctly.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:h1_evaluation)),oql:(and:!((field:!(meta_robots_index,equals,true)),(field:!(status_code,equals,'200')),(field:!(canonical_evaluation,not_equals,not_matching)),(field:!(parsed_html,equals,true))))))&fmt=row_objects
- Title length evaluation: Allows to verify if you have respected the Google guidelines in terms of title length.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:title_length,ranges:!((from:40,name:good,to:70),(name:too_short,to:40),(from:70,name:too_long)))),oql:(and:!((field:!(meta_robots_index,equals,true)),(field:!(status_code,equals,'200')),(field:!(canonical_evaluation,not_equals,not_matching)),(field:!(parsed_html,equals,true))))))&fmt=row_objects
- Description length evaluation: Allows to verify if you have respected the Google guidelines in terms of description length.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:description_length,ranges:!((from:110,name:good,to:170),(name:too_short,to:110),(from:170,name:too_long)))),oql:(and:!((field:!(meta_robots_index,equals,true)),(field:!(status_code,equals,'200')),(field:!(canonical_evaluation,not_equals,not_matching)),(field:!(parsed_html,equals,true))))))&fmt=row_objects
- Open Graph distribution : The Open Graph allows you to add metadatas to the pages of a website to improve the presentation of a web page when it is shared on a social network (Facebook, LinkedIn or Twitter…) (even if this last one has developped its own one, the Twitter Cards). Officially, Open Graph does not have a direct impact on your on-page SEO and it just seems to be here to optimize your social links.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:ogp_type)),oql:(and:!((field:!(meta_robots_index,equals,true)),(field:!(status_code,equals,'200')),(field:!(canonical_evaluation,not_equals,not_matching)),(field:!(parsed_html,equals,true))))))&fmt=row_objects
- Twitter Cards distribution: Twitter Cards also help to improve your Twitter shares. This protocol allows tweets containing links to display in a richer format, with more content visible and integrated as part of the tweet. As the Open Graph, Twitter Cards are a great way to grow engagement and click-through rate.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:twc_type)),oql:(and:!((field:!(meta_robots_index,equals,true)),(field:!(status_code,equals,'200')),(field:!(canonical_evaluation,not_equals,not_matching)),(field:!(parsed_html,equals,true))))))&fmt=row_objects
Content
- Duplicate content: Show the number of pages with content duplication issue: pages whose text content is considered duplicated, or very similar to another page. Duplicate content has a negative impact on a website ranking and the crawl frequency
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:cluster_canonical_status)),oql:(and:!((field:!(meta_robots_index,equals,true)),(field:!(status_code,equals,'200')),(field:!(canonical_evaluation,not_equals,not_matching)),(field:!(parsed_html,equals,true))))))&fmt=row_objects
- Structured datas distribution: Page’s structured datas help search engines to understand what your content is about and contribute to the display of rich snippets in SERPs While there is no direct correlation between those data and a ranking improvement, structured datas can really help you boost your visibility in SERPs.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:semantic_types)),oql:(and:!((and:!((field:!(meta_robots_index,equals,true)),(field:!(status_code,equals,'200')),(field:!(canonical_evaluation,not_equals,not_matching)),(field:!(parsed_html,equals,true)))),(field:!(semantic_item_count,gt,'0'))))))&fmt=row_objects
Performance
- HTTP status code distribution: For each HTTP request, the web server sends a response as a status codes. The main ones are:
- 200: success
- 301 or 302: redirections
- 404: Error page does not exist
- 500: server error
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:status_code)),oql:(field:!(depth,has_value,''))))&fmt=row_objects
- Loadtime distribution: Time to fetch the entire HTML of the page (excluding external resources). Page speed is regarded as a ranking factor.
https://app.oncrawl.com/api/v2/data/project/<project_id>/last_crawl/pages/aggs?aggs=!((fields:!((name:load_time_range)),oql:(and:!((field:!(meta_robots_index,equals,true)),(field:!(status_code,equals,'200')),(field:!(canonical_evaluation,not_equals,not_matching)),(field:!(parsed_html,equals,true))))))&fmt=row_objects
Privacy
Activating this partnership activates the import of Oncrawl data into AT Internet tools and the export of AT Internet data into Oncrawl. Only the data necessary for the proper functioning of the bridge are imported and exported.