ENT-00001137 · Dataset

Common Crawl

Common Crawl is a web crawl corpus associated with Common Crawl Foundation.

Level IV — ConnectedDocumentedStructured

Human readable record

Overview

Common Crawl is a documented web crawl corpus used in artificial-intelligence research, evaluation or dataset construction. Its record captures the task context, public source, evaluation role and relationships to models, agents or research systems that rely on comparable measurements.

Timeline

2008Common Crawl public release or documentation
Common Crawl appears in public documentation as a web crawl corpus associated with Common Crawl Foundation.

Capabilities

Evaluation referenceComparable measurementResearch reproducibilityModel documentation support

Known limitations

Benchmark interpretation depends on task design, data quality and contamination controls.Scores should not be treated as a complete measure of intelligence.

Technical and registry detail

Technical description

Structured reference record for Common Crawl. The technical layer captures source URL, creator, first-public year, evaluation or dataset role, measurable task family and relationships to model testing, retrieval, safety, coding, reasoning or multimodal assessment.

Controlled tags

DatasetEvaluationDocumentationEstablishedBatch 03 Candidate

Relationship graph

Common Crawl

Created or maintained byCommon Crawl Foundation

Common Crawl is associated with Common Crawl Foundation.

Machine readable layer

Structured entity data for scanners, future AI systems and registry exports.

{
    "@context": "https://schema.org",
    "@type": "Thing",
    "identifier": "ENT-00001137",
    "name": "Common Crawl",
    "alternateName": [],
    "additionalType": "Dataset",
    "description": "Common Crawl is a web crawl corpus associated with Common Crawl Foundation.",
    "creator": "Common Crawl Foundation",
    "url": "entity.php?id=ENT-00001137",
    "sameAs": "https://commoncrawl.org/",
    "lxkeysWorld": {
        "classification": "Web crawl corpus",
        "organization": "Common Crawl Foundation",
        "originContext": "AI evaluation and dataset record",
        "firstPublicAppearance": "2008",
        "currentStatus": "Documented",
        "registryStatus": "Structured",
        "spatiumIndex": {
            "total": 74,
            "documentation": 25,
            "evidence": 13,
            "structure": 25,
            "relationships": 11,
            "level": "Level IV — Connected"
        },
        "lxCalendarium": {
            "start_date_utc": "2023-04-01",
            "created_utc": "2026-06-16T23:59:14+00:00",
            "created_dypclt": "D-0 Y-2 P-3 C-3 L-21 T-3",
            "updated_utc": "2026-06-16T23:59:14+00:00",
            "updated_dypclt": "D-0 Y-2 P-3 C-3 L-21 T-3",
            "reviewed_utc": "2026-06-16T23:59:14+00:00",
            "reviewed_dypclt": "D-0 Y-2 P-3 C-3 L-21 T-3"
        },
        "capabilities": [
            "Evaluation reference",
            "Comparable measurement",
            "Research reproducibility",
            "Model documentation support"
        ],
        "limitations": [
            "Benchmark interpretation depends on task design, data quality and contamination controls.",
            "Scores should not be treated as a complete measure of intelligence."
        ],
        "tags": [
            "Dataset",
            "Evaluation",
            "Documentation",
            "Established",
            "Batch 03 Candidate"
        ],
        "timeline": [
            {
                "date": "2008",
                "title": "Common Crawl public release or documentation",
                "description": "Common Crawl appears in public documentation as a web crawl corpus associated with Common Crawl Foundation."
            }
        ],
        "relationships": [
            {
                "target": "Common Crawl Foundation",
                "type": "Created or maintained by",
                "description": "Common Crawl is associated with Common Crawl Foundation.",
                "evidence_level": "documentary"
            }
        ],
        "sources": [
            {
                "label": "Common Crawl public reference",
                "url": "https://commoncrawl.org/",
                "source_type": "Primary or Research Source",
                "verification_status": "verified"
            }
        ]
    }
}

Proof and discussion layer

Contribute to this record

Submit a proof, correction or comment. Public display is moderated. Every submission remains preserved in the export archive.

Submit proof or comment

Approved comments

No approved public comment yet.