RoCKIn@Work was focused on benchmarks in the domain of industrial robots. Both task and functionality benchmarks were derived from real world applications. All of them were part of a bigger user story painting the picture of a scaled down real world factory scenario. Elements used to build the testbed were chosen from common materials in modern manufacturing environments. Networked devices, machines controllable through a central software component, were also part of the testbed and introduced a dynamic component to the task benchmarks. Strict guidelines on data logging were imposed on participating teams to ensure gathered data could be automatically evaluated. This also had the positive effect that teams were made aware of the importance of data logging, not only during a competition but also during research as useful utility in their own laboratory. Tasks and functionality benchmarks are explained in detail, starting with their use case in industry, further detailing their execution and providing information on scoring and ranking mechanisms for the specific benchmark.
Part of the book: RoCKIn
The SciRoc project, started in 2018, is an EU-H2020 funded project supporting the European Robotics League (ERL) and builds on the success of the EU-FP7/H2020 projects RoCKIn, euRathlon, EuRoC and ROCKEU2. The ERL is a framework for robot competitions currently consisting of three challenges: ERL Consumer, ERL Professional and ERL Emergency. These three challenge scenarios are set up in urban environments and converge every two years under one major tournament: the ERL Smart Cities Challenge. Smart cities are a new urban innovation paradigm promoting the use of advanced technologies to improve citizens’ quality of life. A key novelty of the SciRoc project is the ERL Smart Cities Challenge, which aims to show how robots will integrate into the cities of the future as physical agents. The SciRoc Project ran two such ERL Smart Cities Challenges, the first in Milton Keynes, UK (2019) and the second in Bologna, Italy (2021). In this chapter we evaluate the three challenges of the ERL, explain why the SciRoc project introduced a fourth challenge to bring robot benchmarking to Smart Cities and outline the process in conducting a Smart City event under the ERL umbrella. These innovations may pave the way for easier robotic benchmarking in the future.
Part of the book: Human-Robot Interaction