As long as an object is “non-luminous” and “passive,” Sony suggests that it could be used as a controller. The figures in the patent show a banana and a pair of oranges as an example of potential peripherals. The patent says that a user could hold the object in front of a camera, which could interpret the object in order to force inputs on it and make it usable in the game.
The print also explains how the technology could generate “an augmented reality image of a non-luminous passive controller” and then detect changes in the pose of the object to accommodate a player’s hand movements and generate overlapping virtual inputs, mentioning a “finger finder.” In the figure below, you can see the X and Triangle buttons superimposed on a banana, as an example of this idea in action:
“Figure 8 schematically illustrates an example of how the non-illuminated object may appear to the user when one or more virtual buttons have been superimposed above the user’s view of the object,” reads -on in the patent. The copy suggests that an image of the banana with virtual buttons could be viewed via a device mounted on the head, an image “overlaid on a transparent screen through which the user is looking” or a video displayed on the same screen as the player looks at. So once the banana is mapped, you will be able to see where you need to push with your fingers for the entrees.
Other implementations include using motion to steer using two oranges as a steering wheel, changing perspective with a home device via generated axes, and even entering a “pause command” by placing a cup on a saucer while the two objects are detected by the technology.Home engineers have used fruit in the past to play games – for example, play Overwatch with a whole bunch of bananas – but this mainly consists in using the electrical conduction of the fruit to generate commands. This is a more in-depth and potentially flexible process.
In other PlayStation patent news, we recently reported on a piece of technology that allow spectators to play with the players in VR. In April 2020, a patent was discovered for a robotic playmate that reacts to your emotions.
Jordan Oloman is a freelance writer for IGN. Follow him on Twitter.