Harvesting Robot
Artist Impression
Sweet-pepper crop
in the greenhouse

Press and media

Can I get and use photos of the robot?

Where can I find detailed public information about Sweeper?

Are there images taken by the robot camera available?

Can we make an appointment to make a video/photo session with the robot in the greenhouse?

Robot technology

What kind of camera are you using?

Why is the camera using flash-light?

Can the robot work in varying light conditions? Can it work day and night?

How many hours per day the robot can work? How often do I need to recharge?

Does the robot perform maturity/quality classification?

Is it possible to harvest different coloured fruit with this robot?

What happens if leaves disturb the image or the grasping?

The peppers fall down a long distance when dropped into the container, is there no fruit damage?

Can the robot perform additional tasks?

How can performance be improved?

What is the level of the human involvement? Operates the robot fully autonomously?

What are the safety requirements? Can humans be in the greenhouse while robot is working?

Can we integrate the robot with existing packaging/sorting equipment?

Do we have to hire any technical staff to operate and maintain the robot? Who can operate the robot? What expertise do I need?

Robot performance

What is the speed? Success %?

How well does the robot cut the peduncles (fruit stems)?

What progress did you achieve in Sweeper compared to Crops?

Economics

How much will the robot cost?

Can I buy the robot?

What are the main bottlenecks for market introduction?

How many robots do I need for 1 ha of greenhouse?

Is there a patent filed for (parts of ) the robot?

Project and coordination

Will there be a follow-up project?

Why did you chose to harvest pepper and not for example tomatoes?

Already more than 15 years ago you showed a cucumber harvesting robot. What happened with it, why is it not on the market?

How much public money did you get/was spent for this project?

What happened to your partner IRMATO?

 

Can I get and use photos of the robot?

A ZIP file with a collection of high-resolution photos of the robot can be downloaded here:  http://sweeper-robot.eu/dissemination/press/press_photos_sweeper.zip .These photos may be used for your own publications if the photo reference: “Source: www.sweeper-robot.eu” is given.

Where can I find detailed public information about Sweeper?

There are two places on the internet to find information about Sweeper:

·         Our main website: www.sweeper-robot.eu.
The dissemination section http://www.sweeper-robot.eu/dissemination-menu provides an overview over all dissemination activities, including, photos of the robot, handouts of the presentations giving during the demonstrations and a list of publication (general press, magazines and scientific papers).

·         Cordis Sweeper project website from EU: https://cordis.europa.eu/project/rcn/194187_en.html
Here you will also find the public deliverables from the project under the link: RESULTS.

 

Are there images taken by the robot camera available?

The Sweeper project resulted in a number of image datasets that are available for download here:  Public image datasets  (http://sweeper-robot.eu/2-uncategorised/55-public-datasets)

 

 

Can we make an appointment to make a video/photo session with the robot in the greenhouse?

No, not really. The research project was accomplished by October 31st 2018. The robot will not be demonstrated in the greenhouse anymore. Please contact the former coordinator This email address is being protected from spambots. You need JavaScript enabled to view it. about details.

 

 

What kind of camera are you using?

The camera is a combined colour (RGB) and time-of-flight (TOF) model. Both methods are using the same optics and sensor chip. The result is a combined colour and distance image (RGB-D). The camera is a for Sweeper custom made prototype model developed by the Swedish company Fotonic. During the Sweeper project the company Fotonic was acquired by another company that has discontinued the development of this camera. In consequence, this camera will never become available on the market.

Why is the camera using flash-light?

The robot is equipped with custom designed very high intensity LED flash light units to suppress as much as possible effects of changing environmental light conditions. Strong flashlight will ensure constant illumination of the images taken by the camera day and night and also during overcast or sunny days. By flashing the LEDs instead of using constant light allows generate a much stronger light pulse. Also the LED modules will not heat up and use much less energy.

Can the robot work in varying light conditions? Can it work day and night?

Yes it can operate in all light conditions. From full daylight to darkness. We have a flash-light that emits a very short pulse and our camera shutter is opened during this pulse only. This blocks the sunlight very effectively.

How many hours per day the robot can work? How often do I need to recharge?

General maintenance (mainly cleaning) and daily recharging of the batteries will be needed. We expect that a robot can operate 20h/day.

Does the robot perform maturity/quality classification?

Yes. The maturity of a detected pepper is evaluated by calculating two features for each detected pepper:
1. Measuring the colour of the fruit (average hue level in the detected region)
2. Measuring shape features of the fruit (perimeter/area ratio).

Pepper fruits are observed slightly from below. If the bottom part of the pepper is mature, it is most likely that the whole sweet pepper is mature. More information to be found in the public deliverable D5.6 (see Cordis webpage).

Is it possible to harvest different coloured fruit with this robot?

During the project we harvested yellow peppers because of practical reasons. The grower that took part in the project (De Tuindershoek) grows yellow peppers only. The robot can in principle harvest peppers of different colours (red, orange) by simply adjusting the target colour in the detection algorithms. However, the robot was not yet tested for this. To be able to let the robot harvest green peppers, the detection routines must be adapted to detect the green peppers in the green crop by for example using deep-learning image analysis techniques or special hardware like multispectral cameras. This was not tested, but it is expected to be a viable solution with the developed knowledge

What happens if leaves disturb the image or the grasping?

Image disturbance: The prototype will not see and not harvest the pepper and move on to the next detected pepper. For this reason, performing some leaf picking before harvest increases the harvest success. We call this modification the “modified crop”. In future we expect that a crop variety will be bred that have a better pepper visibility.

Grasping disturbance: Currently, almost all peppers cut will fall down into the finger type catching device. However, while approaching the pepper, the fingers or other parts of the end-effector may push forwards the target stem, and in such cases the pepper cannot be reached by the cutting knife. This affects currently the harvest performance of the prototype. We expect to mend this fully by a re-design of the catching device and by reducing the overall dimensions of the end-effector.

The peppers fall down a long distance when dropped into the container, is there no fruit damage?

During testing we did not observe fruit damage caused by this operation. However, dropping the fruits in the blue small container is a temporary solution. In the final robot concept we use a different method. The peppers will be placed onto a conveyor belt, and then they will not drop over a long distance.  

Can the robot perform additional tasks?

The existing robot prototype was not programmed to do so. However we do expect that this specifically will be the extra value of a robot autonomously working in the greenhouse. By using its already available camera’s and 3D detection equipment it could gather an enormous amount of data about the crop. All this is of value to the grower for his crop management and his yield prediction. In other projects we already work on these concepts and we can not only measure data like fruit size, colour, ripeness and anomalies. We plan to also detect diseases and even internal fruit quality.    

How can performance be improved?

The experiments identified current shortcomings bottlenecks and point out improvements that can still be made to the developed robot. Methods to increase the functioning of each submodule should be further studied. This includes investigating a different type of robot arm, a smaller design of the end-effector, a redesign of the catching device and an faster concept for storing the fruit.

What is the level of the human involvement? Operates the robot fully autonomously?

The existing prototype is operated with human supervision. Once it is brought into the cropping row, it can autonomously advance in the row and can harvest peppers on the left and right side of the row.  In a future fully automated high-tech greenhouse we expect that the robot may work 24/7 autonomously with a little downtime for maintenance, and battery recharging. Effectively, we estimate that it will be able to work 20h per day.

What are the safety requirements? Can humans be in the greenhouse while robot is working?

The existing research prototype has a number of safety features on board. At the current development state it may only be operated by skilled and trained personal, and no humans should be within a distance of about 2 m from the working robot. This means under the existing conditions that no other humans are allowed to enter the next cropping row to the left and right of the one in which the robot is working.

The current machine guidelines do not allow humans to work closely to the robot since it can move too fast and has a too high mass for this. Working in a closed environment however makes a slightly easier to introduce robots than f.i. under outdoor conditions (autonomous cars and precision agriculture).

In future we expect that human-robot collaboration will be a very common approach and a good solution to be able to have profit from both skills from humans and robots. For this the existing guidelines are required to be adapted for these set-ups. The authorities already showed interest to discuss about these options.   

Can we integrate the robot with existing packaging/sorting equipment?

Yes, that is the intention. In fact the company involved in this project (Bogaerts Greenhouse Logistics) already has the basics for such on the market. The step to be solved still is the automatic traffic management for the harvest trolleys and robots, localisation and guidance. Furthermore the fully assurance that the robot will not harm any humans working in the same greenhouse.

Do we have to hire any technical staff to operate and maintain the robot? Who can operate the robot? What expertise do I need?

The number of autonomous vehicles and robots will increase in the greenhouse. The final plan for maintaining those must be made still. We expect that a technical qualified person is needed at each greenhouse site. A helpdesk is needed to give long-distance support worldwide.

 

 

What is the speed? Success %?

The robot was designed for a single row plant stem cropping system using an optimized pepper cultivar for robotic harvesting. As this optimized cultivar is not yet available it was tested in a commercial greenhouse first. For the current double row plant stem growing system 18% of ripe fruit were harvested in commercial crop and 49% of ripe fruit were harvested in the modified crop. For this modification most occluding leaves and fruit clusters were pruned away beforehand. Under a single row plant stem cropping system assumption 29% of ripe fruit were harvested in a commercial crop and 61% in a modified crop. The average cycle time to pick one fruit was 24 seconds. During the greenhouse experiments the robot was not operated at maximum speed for security reasons. Under laboratory conditions we have proofed that the time to pick one fruit is not more than 15 seconds.

How well does the robot cut the peduncles (fruit stems)?

The cutting device makes very clean cuts, very close to the plant stem, often at or close to the so called abscission zone. In experiments we proofed that neither the fruit nor the remaining crop have a higher risk on infections. Further, the quality and shelf life of the picked fruits are not affected.

What progress did you achieve in Sweeper compared to Crops?

In Sweeper we made a great improvement compared to our previous research robot Crops (www.crops-robots.eu). That robot was able to harvest 6% in the commercial, unmodified crop and 33% of the fruit in the modified crop. Furthermore it took the crops robot to harvest a pepper about 94 seconds. With the Sweeper robot we can harvest 29% of ripe fruit in a commercial crop and 61% in a modified crop (under a single row plant stem cropping system assumption – where the robot was designed for). The average cycle time to pick one fruit in the greenhouse with Sweeper was 24 seconds. Under laboratory conditions we have proofed that the time to pick one fruit is not more than 15 seconds.

 

 

 

How much will the robot cost?

Based on a robot that can harvest one pepper per 12 seconds and having a harvest success rate of 95%, our economic analysis showed that the room for investment for one robot is between 75 and 100keuro assuming a payback time of 7 years. However, currently we cannot forecast what the actual cost of a single robot will be. For growers a total solution, with several robots, autonomous ground vehicles (AGVs) and a fully automated logistic system is important, and the total costs should be competitive to the current systems.

Can I buy the robot?

The Sweeper robot is a unique prototype that is the result of a research project. It is not possible to buy (copies of) it. However, we expect that after some further development, commercial pepper harvesting robots will be on the market in about 3-5 years.

What are the main bottlenecks for market introduction?

The crop: Peppers should have long peduncles, little clustering, long internode distances and a good visibility. The results obtained were for a pruned crop by taking away peppers from clusters and some leafs. We expect that in 4-5 years breeders will be able to develop new commercial cultivars more suitable for robotic harvesting in order to be able to achieve a success rate of up to 90-100%.    

Economics: To develop a future single-stem row cropping system that gives while robotized a similar or even higher overall economic revenue compared to the current double stem-row cropping system. We expect that from a very densely planted single stem row system the net crop yield can be slightly higher, which will work positively.

How many robots do I need for 1 ha of greenhouse?

This will depend on the final speed. We currently estimate that 1 robot per ha will be needed.

Is there a patent filed for (parts of ) the robot?

Yes, we do have filed a patent on the end-effector.

 

 

 

Will there be a follow-up project?

The Sweeper project was accomplished by 31st of October 2018. We are currently looking into possibilities to acquire a follow-up project.

Why did you chose to harvest pepper and not for example tomatoes?

For high-tech greenhouses cucumber, tomato and pepper are the most cultivated crops. We already worked on cucumber some 20 years ago. Many activities on robotic harvesting of tomato are ongoing for many years in several countries already. By the time we started the project we believed that pepper would be the biggest challenge, and we decided to go for it just to push science as much as possible forward.

Already more than 15 years ago you showed a cucumber harvesting robot. What happened with it, why is it not on the market?

At the time we delivered this robot we found out that the market for high-wire cultivation technology was not yet mature. Also the need for automation was not yet so high. As such, at that time there was no real business case for a cucumber harvester. We do believe however that this has changed nowadays. With the developed building blocks from Sweeper, we think it is feasible to get a cucumber harvester onto the market in a relative short time.

How much public money did you get/was spent for this project?

The EU budget used within Sweeper is about 4 million Euros. Additionally to that approximately another 10% came from National public funds.

What happened to your partner IRMATO?

Our previous partner IRMATO got bankrupted half-way our project, and stopped working within our project from March 2016 on. With IRMATO we were able to develop and test a first prototype we called the basic system. However, the current prototype we have (so called the advanced system) is completely built around the existing greenhouse technologies like the harvesting trolley and a mobile platform from our new partner Bogaerts Greenhouse Logistics. We further completely redesigned the end-effector concept.