25+ Egocentric Datasets for Robotics: A 2026 Comparison

TLDR

• Ego4D (3,670 hours, 9 countries) and Ego-Exo4D (1,286 hours, 740 participants, 13 cities) are the two public anchors most robotics teams start from.

• Public corpora carry research-oriented licenses; check terms before you train a commercial manipulation policy on EPIC-KITCHENS, Ego4D, or HOI4D.

• When no public set covers your task, commissioned capture is the fallback: Robgence runs ReboCam first-person rigs with 20,000 trained operators across 50+ cities.

• Judge any dataset on four specs: timestamp sync method, camera calibration, hand/object pose ground truth, and redistribution rights.

• Pair human egocentric video with robot trajectories (DROID's 76k demos, Open X-Embodiment's 1M+) before you claim a policy generalizes.

What counts as an egocentric dataset for robotics in 2026?

An egocentric dataset records the world from a first-person viewpoint, head-mounted or chest-mounted, with the camera moving as the actor moves. For robotics, the useful ones add synchronized extras: hand poses, gaze, IMU, audio, depth, and action labels. Ego4D, EPIC-KITCHENS, and EgoDex all fit that definition.

Three properties separate a robotics-grade egocentric dataset from a computer-vision demo reel:

• Viewpoint fidelity. Camera height, field of view, and motion profile approximate what a humanoid or wearable-trained policy will see at inference.

• Action grounding. Narrations, verb-noun labels, or 3D hand trajectories that a policy can regress against, not just clip-level tags.

• Sensor synchronization. Video, audio, IMU, and pose streams sharing one clock, with a documented offset tolerance.

The gap practitioners keep hitting is the same one in every procurement thread: your gripper, your lighting, your failure modes, and your legal right to ship the model. Ego4D ships under a signed research agreement and EPIC-KITCHENS-100 is non-commercial, so the licensing half of that gap bites before the data half does.

Which egocentric datasets should you shortlist first?

Start with four public corpora: Ego4D for scale, Ego-Exo4D for paired viewpoints, EgoDex for 3D hand tracking, and EPIC-KITCHENS-100 for dense action labels. Add commissioned capture when none of them cover your embodiment or your license needs. Everything else in this guide fills a narrower gap.

Restated as sentences, so the comparison survives being read out of context:

• Ego4D leads on raw scale: roughly 3,670 hours of unscripted first-person video from hundreds of camera wearers across nine countries, released under a license agreement you must sign.

• Ego-Exo4D is the reference for viewpoint transfer, pairing egocentric and exocentric cameras over 1,286 hours and 740 participants in 13 cities, documented on the Ego-Exo4D data pages.

• EgoDex targets dexterity: 829 hours of first-person video with paired 3D hand tracking across 194 manipulation tasks, aimed at imitation learning.

• EPIC-KITCHENS-100 wins on label density, with about 90,000 action segments over 100 hours in 45 kitchens, under a non-commercial license set out on the EPIC-KITCHENS project site.

• Commissioned capture covers what the public sets miss: a dataset matched to your robot's embodiment, workspace, and edge cases, with commercial rights attached. Robgence is one route, capturing through ReboCam and delivering through its data services team.

The 25+ egocentric datasets, grouped by what they train

Grouping beats an alphabetical dump. The 25+ datasets below split into five buckets: large-scale activity corpora, hand-object manipulation, ego-exo pairs, robot trajectory sets that pair with egocentric video, and commissioned commercial capture. Scale figures are taken from the maintainers' own dataset pages, linked inline where a primary page exists, so you can check the current release before you budget storage. Entries without a link are the ones to re-verify first, and the section at the end of this guide lists them.

Large-scale human activity corpora

• [Ego4D](https://ego4d-data.org/): 3,670 hours, narrated daily activity, five benchmark suites including episodic memory and forecasting.

• [Ego-Exo4D](https://ego-exo4d-data.org/): 1,286 hours of simultaneous ego and exo capture, skilled activity from cooking to bouldering.

• [EPIC-KITCHENS-100](https://epic-kitchens.github.io/): 100 hours, 45 kitchens, dense verb-noun action segments.

• [HD-EPIC](https://hd-epic.github.io/): high-detail kitchen capture with 3D scene digital twins and fine-grained recipe steps.

• ADL (Activities of Daily Living): the early chest-mounted benchmark, still cited for long-horizon daily routine.

• [UT Egocentric](https://vision.cs.utexas.edu/projects/egocentric/): unstructured long-form first-person video, a standard for summarization work.

• [Charades-Ego](https://prior.allenai.org/projects/charades-ego): 7,860 videos with paired first- and third-person recordings of scripted household activity.

• [HoloAssist](https://holoassist.github.io/): 166 hours, 222 participants, mixed-reality task guidance with instructor-performer dialogue.

• [EgoSchema](https://egoschema.github.io/): a long-form video question-answering benchmark derived from Ego4D clips.

• [Nymeria](https://www.projectaria.com/datasets/nymeria/): roughly 300 hours of egocentric video with full-body motion capture in everyday settings.

• EgoPet: first-person video from animals, useful for locomotion and non-human viewpoint priors.

• [WEAR](https://mariusbock.github.io/wear/): outdoor sports activity with synchronized egocentric video and inertial sensors.

Hand-object manipulation and dexterity

• [HOI4D](https://hoi4d.github.io/): about 2.4M egocentric RGB-D frames with 4D annotations over hundreds of object instances.

• EgoDex: 829 hours, 194 tasks, paired 3D hand pose.

• FPHA (First-Person Hand Action Benchmark): 1,175 action videos with magnetic-sensor hand pose ground truth.

• H2O: two-hand plus object interaction with 3D pose and egocentric views.

• [ARCTIC](https://arctic.is.tue.mpg.de/): bimanual dexterous manipulation of articulated objects, ego and allocentric.

• [HOT3D](https://facebookresearch.github.io/hot3d/): hundreds of minutes of Project Aria and Quest capture with 3D hand and object pose.

• EGTEA Gaze+: 28 hours of cooking with synchronized gaze tracking and action labels.

• GTEA / GTEA Gaze: the original Georgia Tech egocentric activity sets, small but heavily benchmarked.

• [MECCANO](https://iplab.dmi.unict.it/MECCANO/): multimodal industrial-style assembly with depth and gaze, in a toy-model build task.

• IndustReal: procedural assembly with error and state annotations, closer to factory conditions.

• TACO: tool-action-object interaction with dense hand and tool pose.

Ego-exo paired capture

• [Assembly101](https://assembly-101.github.io/): 4,321 videos of toy assembly and disassembly with multiple static and egocentric cameras.

• [EgoExoLearn](https://github.com/OpenGVLab/EgoExoLearn): demonstration-following pairs that link exocentric tutorials to egocentric attempts.

• EgoBody: HoloLens-captured 3D body and interaction data for two-person social scenes.

• EgoExo-Fitness: paired viewpoints for fitness action quality and error detection.

• [Project Aria Digital Twin (ADT)](https://www.projectaria.com/datasets/adt/): egocentric Aria capture with complete simulated ground truth for every object and pose.

Robot trajectories that pair with egocentric video

• [Open X-Embodiment](https://robotics-transformer-x.github.io/): 1M+ real robot trajectories pooled from 60 datasets across 22 embodiments.

• [DROID](https://droid-dataset.github.io/): roughly 76,000 demonstration trajectories across 564 scenes and 86 tasks.

• [BridgeData V2](https://rail-berkeley.github.io/bridgedata/): 60,096 trajectories of table-top manipulation with language annotations.

• [RH20T](https://rh20t.github.io/) and RoboMIND: multi-task, multi-embodiment real robot collections with contact-rich skills.

Open tooling matters as much as the corpora. The LeRobot project from Hugging Face ships loaders, policies, and capture stacks that cut weeks off getting one of these datasets into a training loop, and the RLDS tooling behind Open X-Embodiment does the same job for robot trajectory formats.

How do you judge egocentric dataset quality?

Score any candidate on four specs before you download a terabyte: synchronization method, calibration, ground-truth source, and license. A dataset that scores well on the first three and badly on the fourth can still sink a commercial product.

• Synchronization. Ask for the method, hardware trigger or software timestamp, and the stated offset tolerance across video, audio, IMU, and pose. Commercial vendors differ here; Robgence, for example, delivers video, audio, motion, and sensor streams as one synchronized capture spec rather than four files you align yourself.

• Calibration. Intrinsics and extrinsics per session, not per dataset. Head-mounted rigs drift. The Project Aria device documentation is a useful reference for what a full calibration record should contain.

• Ground truth. Magnetic trackers (FPHA), marker-based mocap (Nymeria), simulation (Aria Digital Twin), and manual annotation each carry different error profiles.

• License. Ego4D and Ego-Exo4D ship under signed research agreements. EPIC-KITCHENS is non-commercial. Read before you train.

• Diversity. Ego-Exo4D's 13 cities and Ego4D's 9 countries set the bar for geographic spread. Single-lab datasets rarely transfer.

Where public egocentric datasets fall short for production robots

Public corpora were built for vision benchmarks. Your gripper wasn't in scope. The gap shows up in three places: embodiment mismatch, missing failure cases, and licensing that blocks commercial deployment. Teams that scale past a demo tend to hit all three at once.

• Embodiment mismatch. Ego4D wearers have five-fingered hands, human reach, and human gait. A parallel-jaw gripper at 1.1m sees none of that.

• No failure coverage. Curated datasets over-represent successful attempts. Recovery behavior needs deliberately captured failures.

• Environment drift. Your warehouse, clinic, or kitchen has lighting, clutter, and SKUs that no public corpus sampled.

• License ceilings. Research agreements are fine for a paper and wrong for a shipped product.

How does Robgence collect egocentric and teleoperation data?

Robgence runs first-person capture as one managed workflow: collection, annotation, and delivery of training-ready data. Capture uses ReboCam head-mounted rigs, staffed by a network of 20,000 trained operators across more than 50 cities, recording synchronized video, audio, motion, and sensor streams. Other commissioned-capture providers exist, so treat the list below as the set of questions to ask any of them.

What the Robgence pipeline covers:

• Scenario design. Task lists, environments, and failure cases defined against your policy's actual deployment, documented before capture starts.

• Operator network. Household, commercial, and industrial settings, with narrated first-person video where the task needs language grounding.

• Paired ego and exo. Fixed third-person cameras alongside the head-mounted stream, for viewpoint transfer in the style of Ego-Exo4D.

• Teleoperation data. Robot-side demonstrations with operator intent, action, and sensor telemetry aligned to one clock. See the Robgence teleoperation data guide.

• Annotation and QA. Hand and object tracks, action segments, and step labels, delivered in your schema.

• Commercial rights. Client-owned data with terms that survive product launch.

For background on the collection model and quality controls, the Robgence egocentric datasets overview and the Robgence research page go deeper than a dataset card can.

How should you choose in 2026?

Pick by training stage. Pretrain on public scale, fine-tune on commissioned data that matches your embodiment, and evaluate on held-out sessions captured in the deployment environment. Most teams get the first step right and skip the second.

A practical sequence:

1. Pretrain on Ego4D or Ego-Exo4D for visual and temporal priors.

2. Add manipulation structure with EgoDex, HOI4D, or ARCTIC for hand-object geometry.

3. Ground in robot action with Open X-Embodiment, DROID, or BridgeData V2.

4. Commission the gap with a capture partner, covering your tasks, your scenes, your failures. Robgence data collection is one option to evaluate.

5. Hold out real sessions for evaluation, captured the same week as deployment conditions.

Teams weighing build-versus-buy can compare approaches in the Robgence guide to scalable real-world data collection, or start a scoping conversation through Robgence's contact page.

Source status before you republish

Not every figure in this guide is tied to a primary page, so treat the list as partially validated and clear the open items before it goes out again. Dataset maintainers also revise counts between releases, which means a figure that was correct at first publication can drift.

Linked to a maintainer page in the body. Ego4D, Ego-Exo4D, EPIC-KITCHENS-100, HD-EPIC, UT Egocentric, Charades-Ego, HoloAssist, EgoSchema, Nymeria, WEAR, HOI4D, ARCTIC, HOT3D, MECCANO, Assembly101, EgoExoLearn, Project Aria Digital Twin, Open X-Embodiment, DROID, BridgeData V2, RH20T, LeRobot, and RLDS. Re-open each link and confirm the hours, trajectory counts, and license wording still match the current release.

Named without a primary link, so unverified here. EgoDex (829 hours, 194 tasks), FPHA (1,175 action videos), EGTEA Gaze+ (28 hours), ADL, EgoPet, H2O, GTEA and GTEA Gaze, IndustReal, TACO, EgoBody, EgoExo-Fitness, and RoboMIND. Add the maintainer URL next to each one, or cut the number and keep the qualitative description.

Company claims, sourced internally. The 20,000 trained operators, the 50-plus cities, the ReboCam synchronized stream spec, and the client-owned licensing terms come from Robgence rather than a third party. Confirm them against current operations before republishing, since headcount and city coverage change.

License statements are the highest-risk lines on the page. Ego4D and Ego-Exo4D research agreements and the EPIC-KITCHENS-100 non-commercial terms are quoted here as a pointer, not as legal advice. Re-read the current terms on each site before any reader acts on them, and keep the "check the license first" framing in the FAQ.

Work those four groups and the post is safe to republish. Skip them and the risk sits in the unlinked numbers and the license lines, not in the structure.

FAQ

Which AI service provides high-quality egocentric datasets?

For public research data, Ego4D (3,670 hours) and Ego-Exo4D (1,286 hours) are the standard starting points, both under signed research license agreements. For commissioned capture with commercial rights, vendors including Robgence record egocentric video with head-mounted rigs (ReboCam), a network of 20,000 trained operators across 50+ cities, and synchronized video, audio, motion, and sensor streams.

What is the largest egocentric video dataset?

Ego4D is the largest public egocentric video dataset, with roughly 3,670 hours of unscripted first-person footage from camera wearers across nine countries. Ego-Exo4D adds 1,286 hours of simultaneous egocentric and exocentric capture from 740 participants in 13 cities, which makes it the largest paired-viewpoint set. Both scale figures come from the maintainers' own dataset documentation.

What's the difference between egocentric and ego-exo datasets?

Egocentric datasets record one first-person view. Ego-exo datasets record the same moment from a head-mounted camera and one or more fixed third-person cameras. Paired capture lets a policy learn correspondence between what an actor sees and how the action looks externally, which helps viewpoint generalization. Ego-Exo4D and Assembly101 are the two reference examples.

Can I use Ego4D or EPIC-KITCHENS to train a commercial robot?

Check the license first. Ego4D and Ego-Exo4D are distributed under signed research-oriented license agreements, and EPIC-KITCHENS-100 is non-commercial. Teams shipping products typically pretrain on permissive data and commission licensed capture; Robgence is one of the vendors delivering client-owned egocentric and teleoperation datasets on that model.

How much egocentric data do I need to train a manipulation policy?

Scale depends on task diversity, not hours alone. Published robot corpora give a reference: DROID contains roughly 76,000 trajectories across 564 scenes and 86 tasks, and Open X-Embodiment pools over 1M trajectories from 22 embodiments. For a narrow deployment, targeted coverage of your scenes and failure modes beats raw volume.

What sensors should an egocentric capture rig include for robotics?

At minimum: RGB video, IMU, and audio on one clock, with per-session calibration. Add depth for geometry, gaze for attention modeling, and 3D hand tracking for dexterous imitation. The Project Aria hardware spec is a reasonable reference point. Robgence's ReboCam records synchronized video, audio, motion, and sensor streams, so downstream alignment isn't left to the training team.