This is best described as a fitness rhythm game closely losely a circle gym or a calisthenics workout.

During the last months, and due to the need to find a way to motivate myself to exercise more while staying at home, I developed my hand tracking fitness experiment for the Oculus Quest into a relatively feature complete (at least for a v1.0) workout game with exercises for the core, legs and arms and custom song support.

For PC VR it supports VIVE trackers or otherwise "simply" strapping the controllers to the wrists

It is developed with the Godot engine which has come quite a long way since the VR support has been introduced.

Kudos to the Godot developers and especially Holger Dammertz for his relentless work on the Godot Oculus Mobile plugin

The game is opensource (although the code is still a mess) and also free on Steam or Sidequest

The best part of writing this was and is the feedback of others who are using it and getting a health benefit out of it.

This is cool. I own a older rift 2 (the kind with the actual trackers).

I was wondering if anyone has figured out how to put some sort of stickers on kettlebells or dumbbells that would allow for outside tracking? I think it would be pretty nice if it could show me a guided line of how I should move a weight through space, and then just let me attempt to move it through said line.

I think it would also be neat for tracking your actual sets and reps and basically acting as a personal instructor in terms of keeping you focused.

For the Vive series, there are external trackers that can simply be attached to any object:

https://www.vive.com/us/accessory/vive-tracker/

These obviously require some way to fix the tracker to the object, but otherwise I don't see any technical reason it couldn't be done.

The more practical issue is simply that you're now moving a heavy object without being able to visibly see where it is relative to yourself, the real floor, or any stationary objects in the room. If you drop it, or anything at all glitches with the tracking, it's a safety risk. I agree with the other commenters here; I'd wait to even attempt this until we have something closer to real consumer AR. The restricted visibility in current VR is not worth taking that chance.

I am not familiar with the dumbbell exercises but I would be a bit worried by moving a heavy metal object around without actually seeing what's in the space. I think this would be a better fit for AR glasses but I am pretty sure that such things are coming in the not too distant future since fitness in AR/VR is already starting off to a good start

I’ve been wanting something just like this. Thanks and thanks for providing source as well.

It's a shame that playstation tries to wall off their development support. This would be great on the PSVR.

Awesome! I've seen hackathon implementations of this idea, but yours is the best I've found so far. I'm curious about automatic activity recognition. Will be interesting to train a classifier on head and hand pose to count sit ups or pushups automatically. Might hack around with this later in the month. Thanks for MIT licensing it!!!

Thank you, I am glad that you like it. In the beginning I was thinking about recognizing the exercises but after a while I got comfortable with the motion being driven by the hand and head cues and it being just in between a game and a plain exercise. The cues that are flying at you are just enough distraction from the work you are doing that it pushes the motivation a bit but not enough to be too distracting since you still have to focus on the motions to not hurt yourself.

I do still intend to add some "intelligence" into it to at least track the form of the exercise but that is far down the roadmap

Great work! I always think how we can help senior citizens to simple stretching and fitness exercises at home.

One challenge for them would be to wear the headset, because it may be heavy or cumbersome.

I am sure something can be worked out ...

Some food for thought here!

I feel that the biggest challenge is this: What question is VR trying to answer in this regard? What problem does this solve?

What are the impediments that keep seniors from exercising? Is the fitness market too much geared towards young bodies? Is it solving a mental hurdle to get them to exercise? Which hurdle is this? A fear of confronting the increased risk of injury? Re-connecting with (younger) people? Is it about the cost of entry? What kind of senior are you targetting? At 70, you'll have very fit seniors, and you'll have people dealing with a wide variety of health issues. What about price and affordability? What are alternatives that don't involve complex technology?

So, what does VR do? It's a medium. But like any medium, the real value is in the content you offer, and you need to keep that content fresh and engaging. How are you going to differentiate from personal trainers on DVD, television, YouTube,...?

My PT taught me a ton of simple exercises after a bad injury. There's a ton you can do with a simple elastic band and your own body weight. The most important part is to get into a routine and turn it into a habit. So, the technology shouldn't be detracting attention away from the exercise itself.

Motivation is the biggest challenge. You remember Nintendo's Wii Fit? It was awesome! Nobody uses it / talks about it anymore. That's part of the problem: there's always current tech and the expectation of the "next big thing" that draws attention away. Whereas doing exercises in later life is a sustained habit you hope to teach people across multiple years, exceeding the shelf life of technology.

At the moment the exercises are quite intense (or at least fast) so they may not all be suitable for senior players but I do already have slower exercises in the pipeline where the player needs to hold a cue and follow through with the motion (think of yoga and warmup exercises)

As for the headset. I already am used to it but I hope that the next gen headset will address the fitness aspect as well.

I am actually also eyeing those AR glasses (like the Nreal). The game mechanic and graphics are simple enough that it should be a straight forward translation, the only thing I am worried about is the narrow field of view, but even that could work since the exercises are all focused as coming from the front of the player.

Tracks your running speed in a 4 square meter area? Running in place?

Ah, yes. While the Quest is wireless it still would be a short run :) It's similar to the knee high running exercise one can do when running in place but I can't track that yet.

Well, I have been doing my workout in VR for the past 6 months. So far I am quite happy with it, which is why I continued developing this. It definitely might not be for everyone but the feedback I have gotten so far is really positive.