I don't understand why this entire field seems so focused on gaming, and not productivity. The single most interesting thing to me was the "displays on demand".
There are two things that would absolutely revolutionize how I work that I would pay big money for:
1) Any sort of glasses that would allow me to view virtual displays in high definition. I don't care if I have to turn my head to see more than one display, I don't care if they are VR instead of AR. Should be high enough resolution to be able to use busy Excel spreadsheets and see enough details on a page to do web development
2) Some sort of glove where I could move my fingers to type. It doesn't need to represent an actual keyboard. I could learn whatever new gestures are required for each character.
Those two innovations would mean freedom for me. You could effectively work anywhere in any position, laying in a hammock, on a crowded train, at night in bed when inspiration hits without waking your significant other. It would have to be AR to use while running or working out :)
If these can be made with enough quality to enable equal productivity to a laptop the creators will have an addressable market of about 3 billion people.
There are two reasons why the field is focused on gaming and not productivity: fidelity and familiarity. It's hard to build high-enough fidelity into the hardware at a reasonable price point. And even if you did make VR goggles that were great for giant spreadsheets at $1000, it would be a weird enough idea that it would have to be massively better than $1000 worth of monitors to get people interested.
In gaming, on the other hand, the unfamiliarity of the tech is not a risk but an asset, making the experience more novel. The fidelity doesn't have to be sufficient to overtake an existing process, just to support a fun experience. That's a more scalable business with lower technological barriers to entry, so that's where businesses are focused.
Same reason we had Pong before PCs. You might also see major tech advances in products for specific targets, like military use, medical, or advanced manufacturing, that then trickle down to mainstream productivity applications. But until they get the tech good and cheap enough, expect progressively better games!
> It's hard to build high-enough fidelity into the hardware at a reasonable price point. And even if you did make VR goggles that were great for giant spreadsheets at $1000, it would be a weird enough idea that it would have to be massively better than $1000 worth of monitors to get people interested.
I doubt that. If you make a VR/AR/?R version of a Bloomberg terminal or Factset then finance firms will literally roll up wads of cash and throw them at you. It doesn't have to be massively better. It just can't be worse which is the real hurdle. Granted I'm not an expert on the subject but every VR/AR app I've seen that claims to be the "$APP Killer" simply sucks.
This is the same reason nobody has dethroned Excel in finance. There's plenty of things that solve specific sub problems but the generic, "I have tabs of data and I want to slice and dice it" always goes back to an analyst exporting data to an Excel file.
I've never worked at a financial firm but I have a hard time imagining the uptight social culture of finance being one that embraces employees being spotted wearing goofy headsets and waving their hands around. The boost in productivity from any such app would seemingly be negligible, especially when you consider that the popularity of the Bloomberg terminal lies in its chat function:
* Impromptu breath holding contests complete with prop bets and name chanting
* Chairs shoved, phones slammed, mice thrown, (full) drinks thrown, (Other people's) paper piles toppled, soooo much profanity.
* One time a trader yelled FKING FK very loudly as our EVP of Trading was giving a tour of the office to a business journalist. To which the EVP turned to the journalist and very calmly said, "With us, you always know where you stand."
I worked on two different (energy) trading floors. One was a fairly large venture (30 or so trading desks + mid/back office + dev/ops/DC teams + management). The other was one of the huge banks. Both were in Stamford CT.
I spent well over 5 years there and never once did I see those sorts of shenanigans.
There was some stuff: the guy that liked to throw a football across the floor, occasionally smashing a monitor; the guys that bet $1000+ each over who could lose the highest % of body weight in 2 months (or was it one month?); and the occasional swearing, but nothing near the "like a sailor" level I hear about.
Pretty much everyone I worked with was highly professional. They were profit motivated (for sure), but even there I personally witnessed people making fair deals where they could have squeezed someone dry and unwinding deals at a loss to keep a good counterparty relationship.
I'm so glad I never worked with the handset smashing, drug abusing, loudly swearing, king of the world, type-A assholes that I hear about.
Been there done that. Was the big bank one, the one with the keys in it's logo ? If so, you were using the Desktop OS build I designed :)
> I'm so glad I never worked with the handset smashing, drug abusing, loudly swearing, king of the world, type-A assholes that I hear about.
I think these days, thats largely confined to the Hedge-Fund traders. And those guys are EXACTLY the sort of people would love AR/VR based trading UIs - anything to make more money is always welcome in their world.
I have a VR system and I wear it knowing that it makes me look like a total doofus. You also have zero ability to discern what's going on around you, e.g. to easily socialize with someone walking by. The productivity advantage a VR application would have to be huge for it to be worth doing things by VR, and the fact is that the fidelity just isn't there yet. I would also argue that even with fidelity, the usecases seem specious. But I also thought that the "Minority Report" scenes (which didn't involve a siloed-VR-helmet experience), while cool-looking, seemed like an extremely inefficient way to solve crimes.
Frat-like behavior on trading floors is for members of the rich kids club or those willing to go through the initiation rituals. The lower-level staffers who support the traders are not invited to participate in such japes and are expected to look and act professional, not to be goofy individualists who have just as much fun at work as the traders.
>>I've never worked at a financial firm but I have a hard time imagining the uptight social culture of finance being one that embraces employees being spotted wearing goofy headsets and waving their hands around.
Put some time in at a long/short or quant fund. This would be par for the course. I've been there.
One important thing about VR goggles is that they are literally for your eyes only. You can be shoulder to shoulder with other people, and be sure they can't eavesdrop.
This can be quite important in certain circumstances.
lol @ uptight social culture of finance. Most trading desks are just ex-frat boys with quant skills or a connected uncle. They're all in on tech gadgets; they all have the latest smartphone / wearables / TV / smart home.
the regulatory compliance of Bloomberg lies in its chat function. seriously, type "sold for 10million" in there and it's a deal, reported to the SEC and sucked into end of day reporting
Some hedge funds do not have any uptight culture and they will most likely readily embrace the goofy glasses and gloves. Way better than a cluster of 6 monitors.
You might be able to collect a few wads of cash from finance firms who want a cooler-looking Bloomberg terminal, but I doubt this strategy could actually displace Bloomberg unless there's a technological edge that (1) delivers real, ongoing value and (2) can't be matched by Bloomberg within 1-2 years.
Problem #1 is what you mentioned with VR/AR $APP Killers - they mostly suck. They create value via novelty, which wears off. Bloomberg delivers value by addressing a really consistent set of needs in a solution highly familiar to many users across a whole industry, to the point it's hard not to buy it if you're in finance. It's like the industry's craigslist - lousy in many ways but good enough to stay entrenched (though Bloomberg's entrenchment is driven more through familiarity and brand rather than network effect, which makes it somewhat easier to compete against).
Problem #2 (being matched by Bloomberg) comes from Bloomberg having such domination of its market, and such deep pockets, that unless you are doing things they can't do they will just learn from and then kill you should you attain sufficient success. Although it might be easier to buy you!
You're not going to displace Bloomberg - the amount, quality and timeliness of the information they provide is on a completely different level (and a completely different business).
You're going to get wads of cash from people eager to consume their Bloomberg data in a better way.
I can assure you, the majority of people who regularly use the Bloomberg terminal are not eager for any change whatsoever to how they consume the data.
Any change, no matter how much it improves the workflow or UI is met with bitter resistance, because the users do everything from muscle memory, and small changes break that and slow them down.
I think this might be one place where that rule is broken. I started seriously using VR a while ago and I was blown away by some of the little things that make it 1000% better than a standard screen for some applications.
The most poignant example I have comes from gaming, but I think it illustrates the point. In Elite: Dangerous, some of your menus are to the left and right of your avatar. Normally, you'd have to press buttons to focus the camera on these menus; in VR all I have to do is look at where that menu is and it pops right up for me to use.
It actually made the interface _way_ more intuitive, easier to use, and significantly faster because it played on my expectation of the result. If something similar could be designed on top of Bloomberg data, I think you'd see finance people trampling each other to get it.
This would be particularly true if you played into their existing behaviors and motions; they wouldn't have to learn anything, it would "Just Work".
I have no trouble believing you. I'm in the financial industry, although as an IT guy and not a finance one, but I gave back that utter abomination of the Lenovo Carbon X1 second generation because of the atrocious keyboard (touchstrip, ESC relocated, Tilde relocated, "split" Del/Backspace key etc).
Same reason I won't buy the new Macbook pro - they fucked with the ESC key, they instantly stopped existing as a viable product for me.
At no point did I suggest messing with their keyboard - just give them better visualization possibilities.
This reminds me of somewhere around 1998. I was on-site technician for a market-data software company that had was complimentary (live charting) to Bloomberg terminals. That year, we were updating our software from DOS to Windows NT. I spent a LOT more time than I ever imagined I would teaching men two to three times my age how to use a mouse. Many did not enjoy the change and most were quite adamant about saying so colorfully.
Those guys could run circles around me in the DOS version with just a keyboard. Seeing them use it now reminds me of watching someone with a decade+ experience with vim or emacs.
It's been a while since I've been in the AR/VR/3D space but last time I was there you just couldn't do high density text well at all.
Look at the effective resolution of your monitor and consider the percentage of the field of view that it occupies. You'd need to get something in the area of 6-8k resolution screens if not more to match the same effective resolution.
Well, the airforce had a laser scanner for their maintenance personal beaming pdf documents directly into the eye. Super sharp. Looked a bit like Google Glasses.
Like in this picture: https://www.x.company/glass/
Yes it can. The resolution of AR/VR glasses is less than my iPhone or my laptop monitor.
Now you’re telling me that plugging in a big bulky headset is somehow going to deliver a not worse experience than thee other devices that don’t require an unparalleled level of immersion (both a pro and a con) yet able to show less detail?
I think he was stating that "can't be worse" was a requirement, not a fact. As in, it _must not_ be worse than existing options ($1000 in monitors), ie: resolution, as easy to do everything you already do, etc.
That's on purpose. Resolution is nothing in VR, reaction time is everything. You only need a very wide field, like 800x200px. This would be luxury VR glasses.
However for other, more non-immersive applications, like AR resolution became more important. Which leads go slower reaction times. Which can lead to sickness and lawsuits.
Another reason is that productivity has very specific integration, security, and deployment requirements before it is used. You would never look at a financial terminal that wasn't secured. A video game on the other hand? Sure, why not?
I’m afraid it’s a fact, the fidelity of current VR/AR tech just isn’t high enough. Money really is being thrown at the problem, but there is no magic wand available at any price. These advances take time.
When I was at <fairly large financial research website company> they had a prototype VR research project that was shown to a few people. Never seemed to gain any traction, and I didn't hear why.
Oh wow, that’s interesting. I thought it was still super common. Do you have a link that I can read to learn more about every bank using custom software?
They aren't all using custom software. But all are using banking software provided by one of the big 4 providers in the industry: FIS, Fiserv, Jack Henry, and D+H
I believe they control 95%+ of the banking market.
Those are valid points, especially your point about fidelity, current tech is good enough for gaming and apparently not good enough for work (or we haven't figured out how to use it right). So they can sell it for that right now.
But this and other comments are vastly underestimating the resources that people would be willing to exchange to make their work better. For what I described, a "reasonable" price point could easily be around $10,000. Apple is selling their new iMac at a starting price of $5,000 and there is nothing particularly revolutionary about it, it's an upgrade of what is already available. When you are spending all day using it, and it is the tool you use to make your money, it is worth a lot.
What is the rate of spending in industry on displays vs computers? My general feel is people skimp on displays and over spend on compute. So it would be hard to fight that trend.
Secondly with VR as a desktop replacement... This is simple math. A 4k (or higher) 30" monitor will always be easier and cheaper to build and worst case the same. Except it's at 3' from your face. Shrink that monitor massively and stick it inches from your face. More expensive. Now do the trig on how many and how small of pixels would be required to pay that same resolution from 3' to 1"... Massive downscale... But great we'll get there on display density.... Except, and this is the kicker... For a lot less you can just upgrade the 4k monitor.
Basically unless something fundamentally changes in display tech or we hit a Max resolution that people care about... Miniture displays will mathmatically lag large displays. Yes smartphones tipped the balance but it all got rolled into desktop displays and tvs.
So you lose on econimics and math.
A display tech with massive resolution but something that inherently keeps it from scaling would be a big kicker. Or as some people have said... Hitting 16k or so resolution on VR screens per eye and your dream is there.
Till then gaming is a massive tech driver... Not people using spreadsheets
I don't need a virtual 4K monitor. I can turn my head to see more, just like I do in real life. How close do you have to be to see the pixels in a 4K monitor, and at that point, can you see the entire monitor without looking around?
I want the portability, and then the resolution just has to be good enough. If we could get a virtual 1080p display that would be amazing and good enough. But to your point, the VR display would probably have to be 4K just to be able to properly represent a 1080p virtual display.
you're not doing the math... this is not meant as a slight but:
how big is your virtual 1080p screen representing? and at what distance. give me those numbers and I can tell you what display tech it needs to be, but it's going to be on the order of 10-20x density. it's all about pixel radians.
Also, 1080 is quite small for reading, macs have retina displays which is nearing print when running anti-aliasing.
And don't think I'm attacking you, I want this too. It's just that I did the math.
I guess I'm ok with a virtual screen taking up most of the view. Of course it would be better if it didn't. But I'd still pay for one where I could pretty much only look at one screen at a time, with enough space around the edges to see if there was another screen to look at. And to avoid eye strain I'd need to not have to focus a few inches away.
So if I could focus like it was 6 feet away but it was big enough it took up most of the screen from that distance, and it had a 1080p resolution, that would be good enough. And then I'd buy the upgrades when they came around :) The rough math I was doing in my head was that anything that wasn't 1:1 would need probably 4 pixels for every one virtual pixel. So 4K would about do it.
I pick 1080p because most laptops today still have that resolution, and several laptops that I still use have less resolution than that and are still usable.
a virtual 1080 display that looks like the real thing would be a big gamechanger. Most companies don't buy 4k displays for their employees because 1080p is generally a lot cheaper and also good enough, even for reading. I still have 2 Dell Ultrasharp 1200p IPS displays at home that are 12 years old but reading/writing text on them is totally fine at the normal viewing distance. I'd want 4K mostly for more real estate.
Just got a $399 50" Samsung TV on BlackFriday. Base on the review, it supports 4:4:4 which is a must as PC monitor. Hook up the one yrs old cheap HP laptop to it via HDMI port.
Looks great! 4k video streaming from Youtube works very good.
4 x 1080 Tiled windows - perfect for productivities.
I don't do gaming. The latency is less of an issue for me.
There is also talk about how we actually only mentally process a small focus point in high resolution, and a renderer that could track our focus point could use a lot less GPU compute for a scene if fast enough.
>And even if you did make VR goggles that were great for giant spreadsheets at $1000, it would be a weird enough idea that it would have to be massively better than $1000 worth of monitors to get people interested.
I'll admit that I might not be the "average person" that this is target, but I completely disagree.
I've often said that developers should have powerful machines and tools, pretty much regardless of price. Salaries are often over $100k/yr, and at that price if something makes you a few percent more productive but costs a few thousand dollars, it will (in theory) pay for itself in less than a year.
And for something like this, it would easily make me more than a few percent more productive. Just being able to take the equivalent of 3 monitors with me when I'm on a plane, or am working somewhere other than my normal workstation alone would be worth it. Plus the fact that i'm no longer limited to screen realestate means that when i'm actually "in the zone" and really getting shit done, I'm not going to be struggling with frustration at needing 3 console windows open, plus 2 editors, 2 browser windows, and a chat window. Just being able to throw a console or 3 at a spot where I can glance without having to take my hands off the keyboard or alt+tab through several windows in some cases would be a giant QoL improvement!
For me, this would truly be revolutionary. A headset that lives up to the promise of "a 1080p screen, but everywhere" would change so much of how I work it would be completely revolutionary. And I'd easily be willing to throw a few thousand at a product like this without even thinking about it (with the caveat that it would need to look like it's going to be supported for a while, and won't just ship a device and never release software for it again).
> It's hard to build high-enough fidelity into the hardware at a reasonable price point.
It's more than that. With the current state of the art, it's impossible to make a "virtual monitor" that can match the display density of even a plain 1920x1080 display. You can make "giant spreadsheets", but only because you have to blow up everything to multiples of normal screen size for text to be readable in the VR environment.
> And even if you did make VR goggles that were great for giant spreadsheets at $1000, it would be a weird enough idea that it would have to be massively better than $1000 worth of monitors to get people interested.
If it took up less space than multiple large monitors, it would already be better by one criterion.
>> You could effectively work anywhere in any position, laying in a hammock, on a crowded train, at night in bed when inspiration hits
> have to be massively better than $1000 worth of monitors
Just read these two quotations, and you have the answer. Yes, high-resolution low-hassle googles would be massively better than 3-4 decent monitors for quite some people. At $500 for 2K resolution, that would be an instant hit.
I think that it's a convenient text input device that is missing from the picture. A lot of power users can't touch-type yet.
Another problem is the hassles: either cables or poor battery life. Bandwidth is important outside of gaming, too, else you'll hear the complaints about janky page scrolling instantly.
The big problem there is resolution and the "Screen door" effect.
Text has to be fairly large in order to be readable. We will need an enormous resolution and incredibly low dot pitch to solve those issues and make productivity tools possible.
This is the tech space where I dream of working. I have been designing features for developer teams in AR but without the hardware being available, it's all theory.
Yup! I cringed when I read the above comment and imagined working in Excel on current AR displays. Resolution is still an issue, as you've stated, and there are still heaps of issues in the realm of focal depths/focal rivalry when it comes to AR. Current displays have a set focal depth and people don't realize how much that can affect things.
There is work[1] to address focal issues, and obviously resolution will continue to get better. I notice the focal issues in VR racing sims when I glance between my mirrors and back to the track, you expect a shift in focal length and it hurts your brain when there is none.
I'm still excited to see where HMDs get to in the next 5-10 years, both AR and VR.
So I haven't tried the tech, but one of the main advantages of the 'light field' tech they've been developing is supposedly solving this focal length issue - i.e. near things appear at a different focal point than objects farther away. Can't say how well it works in practice though.
That and the whole "how do you make stuff opaque". All the demo show semi-transparent object being added to the real world. Making text legible when you can't fully control the background on which it is display is very hard. My best guess is that if they don't advertise that use-case it is because they don't really feel ready to be judged on it.
Magic Leap claims to be able to occlude real-world objects with their AR, not just semi-transparent "hologram" overlays. This is something I'd want to see for myself, but here's what Brian Crecente at Rolling Stone wrote about it:
> Miller wanted to show me one other neat trick. He walked to the far end of the large room and asked me to launch Gimble. The robot obediently appeared in the distance, floating next to Miller. Miller then walked into the same space as the robot and promptly disappeared. Well, mostly disappeared, I could still see his legs jutting out from the bottom of the robot.
> My first reaction was, “Of course that’s what happens.” But then I realized I was seeing a fictional thing created by Magic Leap technology completely obscure a real-world human being. My eyes were seeing two things existing in the same place and had decided that the creation, not the engineer, was the real thing and simply ignored Miller, at least that’s how Abovitz later explained it to me.
If they really have that working, it's a huge advantage over systems like HoloLens.
Based on Magic Leap's explanation, I'm not so sure it would work for that.
> My eyes were seeing two things existing in the same place and had decided that the creation, not the engineer, was the real thing and simply ignored Miller, at least that’s how Abovitz later explained it to me.
That sounds to me like some kind of light-field trickery where it puts an object in front of the background using the light field, but doesn't physically block the light. Instead, your brain processes it out because your visual model of the space has something in front of it.
I'm imagining this works sort of like the effect where you overlay left and right eye images that don't match, your brain sort of fades between the two of them because it can't decide what's there? Except instead of having that disagreement, both of your eyes say "This thing is in front" and that's how you see it"? It's hard to say from the kind of hand-wavey explanation.
If that's the case, I don't know that it would work for really bright glare. Either it might be a strong enough signal to overpower the light field, or the bright light scattering around your eyeball might still cause enough bloom to wash out your vision.
Furthermore, even if the technology DID somehow manage to trick your brain into not acknowledging the glare, the bright light would still be entering your eyes and causing damage to your retinas. It would be similar to the case of somebody who has nerve damage and doesn't feel pain, so they don't know that their hand is resting on a hot stove until they smell burning flesh.
If only we had technology that, when subjected to an electrical field, turned glass from opaque to translucent and were produced in sufficient quantities for applications such as airplane and building windows as to make this technology relatively inexpensive.
The occluder wouldn't be in the same focal plane as the real world objects you are looking at. It'll darken a vague area around the object you want to mask.
Probably I'm missing something but I don't see this as a hard problem to solve because LCDs work by controlling the transparency of a pixel to let trough more or less background light and OLEDs don't need the background light to emit color so combining both you can have controlled transparency behind the color emitting pixels.
Don't you control that by doing text recognition and always placing a frame behind? Let the user customize the color schemes in question (bright white + black text, or off white + off black etc).
If the background is a blue wall in your living room, you place a white framing layer behind the text you're looking at. If the real wall is black, you do the same thing. Makes no difference what color the wall is then.
Wall -> Frame Layer -> Text
Text recognition should be among their easiest chores (which is to say it's still not easy, it's on the lower level of difficulty in what they're trying to do).
The issue is that if you have a transparent display for AR, light from the background goes through it. There's no way to just "put a white framing layer behind it" because the light the display puts out is being added on top of whatever light is coming through from behind it. This is what happens with Microsoft's HoloLens headsets.
Magic Leap is claiming to have occlusion of background objects working, but hasn't really explained the mechanism. It sounds like it's some sort of "light field" trickery where they let the light through, but your brain knows there's a virtual object in front of it and mentally processes it out. Cool if it works, but I'll need to see this to believe it.
Nice. I think the idea is that by tracking and moving with the eye, you don't need a high resolution display. You just focus the resolution you have at the center of the users field of vision where most of our visual acuity is located.
That's a solvable problem though - it's more a matter of cost and the pixel density will go up and the cost down over time. I don't see this a long term limitation of this tech. Have you seen the Pimax 8k VR headset for instance? https://www.kickstarter.com/projects/pimax8kvr/pimax-the-wor...
In reality pimax delivers 1440p per eye. Someone needs to calculate this, but I think even that would be orders of magnitude away from a few 2K+ displays projected in your FOV. I think technology is really far away from that for any price. Don't trust the hype.
It's probably solvable eventually, but the tech isn't there yet.
Companies can focus on the gaming industry first because it's tolerant of slightly degraded looking graphics and inability to render high fidelity text, and they can start selling headsets now to that demographic. A person writing code all day or looking at spreadsheets will not tolerate reading small text through a screendoor.
I see this is a popular wish, and I'm afraid it's frustratingly unimaginative.
Finally we have a medium that can open up entirely new ways of doing things with computers, yet so many people just wish it could replicate the old ways.
Same thing happened when the first display computers were made to emulate paper-based terminals instead of exploring what's possible on a graphical display. We still haven't fully recovered from that.
>Finally we have a medium that can open up entirely new ways of doing things with computers, yet so many people just wish it could replicate the old ways.
Right. But to invent the new ways, I need access to A) a replication of the old ways, B) source/config/whatever so I can evolve the Next Big Thing. Without this use case in mind, we're effectively stuck on keyboards and monitors.
They are explicitly not. They want to abolish the screen. They want real, touchable objects to be imbued with dynamism.
From their FAQ:
"Is Dynamicland augmented reality (AR)?
It depends what you mean by augmented reality. Dynamicland is primarily about working with actual physical objects that everyone can see and touch. Glasses and phone-based AR is usually about 'holograms' floating in space that only the person with the device can see. It is a central tenet that all people who come in to Dynamicland share the same reality. This enables social cues like pointing, eye contact, and shared attention which are essential for people to be fully present with each other."
AR is not VR. I think that VR will certainly enable all these behaviors, eventually. Except in a virtual space, instead of meat space. The behaviors and ideas Dynamicland are advocating should work just as well in both.
And while their points ring true for the current iterations of AR-as-personal-assistant, I don't see why a networked AR where everyone shares the same annotations would work any differently. Isn't that what their projectors are doing, after all?
It's not just the social angle — projections in VR and AR aren't tangible, whereas physical objects imbued with dynamism are tangible. Here's a really good article on why this matters so much: http://worrydream.com/ABriefRantOnTheFutureOfInteractionDesi...
Now, once we have a holodeck, where the virtual objects are made tangible, your points will hold. But until then, tangibility is a key difference (and perhaps not the only key difference, but this should suffice for now).
I don't get the pink plane reference, but this is a good point. I'm conflicted about it. On one hand you are right we should try to use new technology to get to entirely different levels of productivity, opening up "new worlds of ideas".
At the same time, we have built our organizations, relationships, even our own brains to work in a certain commonly accepted way. Everyone can visualize what I mean when I describe a virtual display, which themselves have basically represented virtual pieces of paper as you mentioned. We would be instantly better off. But yes, it would be a tragedy if the better, more revolutionary way of working was discarded for an incrementally better version of what we are used to. I'd hope somehow we could do both.
I believe the pink plane comment is a reference to Alan Kay’s idea that there is an entire “blue plane” of ideas that lie orthogonal to the “pink plane” that people live/think in, that only becomes unlocked with a change in perspective (“looking up”). He talks about it in his How To Invent the Future talk.
> But yes, it would be a tragedy if the better, more revolutionary way of working was discarded for an incrementally better version of what we are used to.
This tragedy is common occurrence with incremental ideas.
Compared to the revolutionary, incremental ideas have an unfair advantage. They are way easier to talk about, way easier to imagine, way easier to implement, and way easier to sell. They capture the market, and before long, too many people's lives will depend on them that no one would dare even propose doing things differently.
This has happened time and time again. QWERTY is a nice example:
> The top row of alphabetic keys of the standard typewriter reads QWERTY. For me this symbolizes the way in which technology can all too often serve not as a force for progress but for keeping things stuck. The QWERTY arrangement has no rational explanation, only a historical one. It was introduced in response to a problem in the early days of the typewriter: The keys used to jam. The idea was to minimize the collision problem by separating those keys that followed one another frequently. Just a few years later, general improvements in the technology removed the jamming problem, but QWERTY stuck. Once adopted, it resulted in many millions of typewriters and a method (indeed a full-blown curriculum) for learning typing. The social cost of change (for example, putting the most used keys together on the keyboard) mounted with the vested interest created by the fact that so many fingers now knew how to follow the QWERTY keyboard. QWERTY has stayed on despite the existence of other, more "rational" systems. On the other hand, if you talk to people about the QWERTY arrangement they will justify it by "objective" criteria. They will tell you that it "optimizes this" or it "minimizes that." Although these justifications have no rational foundation, they illustrate a process, a social process, of myth construction that allows us to build a justification for primitivity into any system. And I think that we are well on the road to doing exactly the same thing with the computer. We are in the process of digging ourselves into an anachronism by preserving practices that have no rational basis beyond their historical roots in an earlier period of technological and theoretical development. – Seymour Papert, Mindstorms
That's why I believe we should be more evangelical in promoting revolutionary ideas (from the blue plane), and more quick to point out incremental ideas (the pink plane).
QWERTY seems like a curious example; I'd hardly call a different keyboard layout a revolutionary idea - neither in its conception nor in its results.
In fact, I'd say it's an example of the opposite: the alternatives didn't surpass QWERTY because they were mere incremental improvements. You need a big leap to beat the inertia of the existing systems.
> I'd hardly call a different keyboard layout a revolutionary idea
Yes. It wasn't my intention to imply that an alternative keyboard layout would be a revolutionary idea. The example was meant to illustrate how a non-revolutionary idea (virtual 2D screens in VR), after establishing itself, can make it difficult/impossible for other ideas to take hold.
> You need a big leap to beat the inertia of the existing systems.
It's good to point, that once an idea becomes mainstream, even research into revolutionary alternatives will dry up compared to those focused on incrementally improving the mainstream, making it even more unlikely that the leap would happen.
Instead of a screen, output used to just be typed on paper like a typewriter. And we have basically emulated that as we worked our way up to modern computers. It's a good point.
> I don't understand why this entire field seems so focused on gaming, and not productivity.
Is it ? If we talk abour VR, yes, but there are a number of companies actually delivering AR products whose focus is on productivity. Some examples (but there are more) are
Microsoft with Hololens [1]
Epson with Moverio devices [2]
Sony [3]
Vuzix [4]
Their oldest model are simple smart glasses, it seems to me that they are moving towards full AR.
As a plus side, they don't look something that I would wear only inside the office of SV startup.
I don't know about the other devices, but the hololens is incredibly underwhelming when used. The marketing CGI for it is the best thing about it. I suspect that is probably true also of the magic leap one.
Have you actually tried it? I know some of the developers and creatives that have worked with it, and they say it's more impressive when used in person than it looks in the 2d promo videos, which is the opposite of most devices.
I have tried it in person; and it was overall disappointing. This was some time last year so maybe they've improved in the mean time but the AR overlays weren't particularly compelling and the FOV was terrible.
I also felt the tracking was mildly worse than Vive; although that is more anecdotal.
I did the Mars demo at a conference a while back - the first couple of minutes I was underwhelmed, but at some point, my brain "bought-in" - I noticed this when I crouched down to look under a virtual rock outcropping without a second thought.
Vuziz is a pretty funny example there, because they used to be (pre-Oculus Rift) focused on VR gaming. The only problem: their product was garbage. I'm astounded the company still exists at all.
I had the chance to try only one of those, so I cannot make an informed comparison: one really has to try the device because I think that there is almost no relation between marketing videos and the actual experience.
My interest is mainly the healthcare sector, and while I have in mind a few applications, I think that for now they are not able to offer something a smartphone in the pocket of a doctor with a properly designed app, cannot do in the same amount of time.
If I had to make a wild bet on what will be the first successful consumer application of AR glasses I would say glasses for cyclists: the user can really benefit from not having to watch away from the road and we are already used to see them riding with funny glasses anyway :-).
(in fact it seems there are already a bunch of companies focusing on that).
Hololens is kinda neat but gesture recognition isn't there yet and the limited FOV really, really sucks. That's the biggest problem with any of these.
The Moverio is setting out to be pretty modest--think "Google Glass screen projected in field of view, but not shit"--but the execution there is solid. It also plays very nicely with glasses.
> I don't understand why this entire field seems so focused on gaming
Because gamers are a community well know to spend several hundreds of dollars easily for any kind of upgrade to their existing hardware. At least the core PC gamers, and that's quite a large segment nowadays.
Myself and millions of others would spend thousands for this, and thousands more for upgrades. When you consider this is the tool that you use to make your salary, anything that makes me more productive with a better quality of life starts to be worth more than the price of a new car.
I think the issue is that you and millions of other more than likely want a lot more assurance it will work well and contribute to their workflow. Gamers are much more likely to make a speculative purchase, or at lease have a smaller set of requirements to consider it a useful purpose.
People's workflow is often fairly static, and this would need to fit the existing requirements. Games are very different from each other and transient. This may not work well for most of a gamer's library, but new games might come out that they will buy and utilize. If this doesn't work well (or even just as well as what you currently use) for a browser, terminal, spreadsheet and text editor, are you going to switch to some new software that you don't know that might work better (but still possibly not as well as what you used originally) just to make use of it? I wouldn't.
Any current workplace that doesn't currently spend $1000 per employee on good, large screens has already voted with their wallet (the most honest way of expressing a true opinion) that they definitely won't consider anything like this to be worth it.
I think your comment illustrates why they should stick to games, but I'm not quite sure how to say it. It has something to do with less pressure, or reducing the requirements or feature creep.
Why do you think your productivity is going to spike? There seems to be enough info being pumped into people's heads as it is, without them actually doing anything useful with it. Sure there is a great appearance of productivity but talk to an economist for the real story.
I think is because of the culture of the company itself, it’s all about creating worlds and leaping towards the next big contract with the user. Those things won’t happen for an Excel user that just want Excel in 3D. Haptic feedback, virtual characters etc. are more gamer friendly
A lot of money is being poured into professional VR and AR applications right now. It does not happen very publicly. I am aware of a few in-house efforts of big manufacturing companies with rather big team sizes (dozens of developers at least). They are mostly centered around stuff where 3D visualization is beneficial and head tracking allows for that extra bit of interaction and exploration. Think "classic" AR ideas such as information overlays (disassembly instructions for mechanics, etc.). Except that this time around they can make the tracking so good that the overlays are really "there".
> I don't understand why this entire field seems so focused on gaming, and not productivity.
Lets look at a similar product that lives on as "productivity", one that did NOT launch with gaming features: google glass.
For most of us, and most companies they aren't going to invest 100's or 1000's of dollars on something unproven. We might do that for "entertainment" purposes but not productivity ones until the tech is proven.
How do you prove out tech? How do you get those consumer grade productivity tools into the hands of end users quickly and efficiently -- lower cost or raise value. There are plenty of folks who gladly spend $1000 on gaming, look at the mobile space and the insane decisions consumers make there.
That was google glasses problem though, they didn't focus enough on productivity. It was a gimmick. Instead of handing it out to techies to play with, they should have made one single useful application, like for doctors or architects, and then only rolled it out to them. Instead of "glassholes" you'd have respected professionals who would come to rely on it for work. More and more applications would be added, and eventually people would use it for gaming and entertainment and daily life. Like, for example, cell phones. And the internet. They both started for productivity purposes and became much more.
And anything that increases productivity will always have plenty of money thrown at it. Think about standing desks, for example. My employer is happy to drop $3,000 on a high end laptop so I can be that much more productive.
Wow, you must work in a magical office. I work in Tech Support for a large company that is in the Fortune 1000, and they don't spend that much for tech. The engineers don't get that much either. They give us hand-me-down laptops in tech support, and buy mid level machines for the software engineers. The will spend more if there is need, but that would be the server, not the user laptop.
Gamers will shellout $1000's though for tech that is limited run, so you can then get items to scale for the majority of businesses/end users to purchase.
Also, Google Glass is being used in productivity, the current gen program is being used only in productivity applications. Google Glass was just too new to the market, and with the camera people were worried about being recorded, it needed a LED or a physical closure for the lens to make people comfortable.
Google glass is absolutely pointless and cannot be used for mixed reality.
It has a fixed small display that forces you to look at there.
Magic leap will allow you to move freely your eyes inside the active field of view.
Beyond these, the main reason is that gaming is a much better exploration and experimentation field than productivity because each app can have far less capital investment, lifecycles are short, and there's no need for a reliable, successful output.
Existing VR displays have your eyes fixed focused either several meters out, or at the horizon, so the kind of eye strain your talking about is not an issue.
MagicLeap supposedly has variable distance focus working.
3D is most obvious when it'd used to visualize physical objects or environments.
Abstract information spaces are hard;
being non-physical and having many more dimensions.
The added interface complexity usually doesn't justify the gain, from only throwing away 23 dimensions, instead of 24.
There might be useful ways to represent clusters of information related to a non-physical task effectively; there hasn't been a clearly successful one yet.
and no, if you are trying to jam a bunch of 2D screens in a 3D environment, you are not "representing physical objects or environments" as identified as the core strength above.
Have you tried out HoloLens, Oculus etc? Technology is simply not there yet. Field of view is very limiting. Resolution is very low. Hardware is highly optimized for graphics, not high performance computation. None of the devices I have tried could be worn for more than 30-45 mins before you start having headache (because your brain is seeing depth but your eyes are on different focus plane). VR/AR tech probably needs 5 more years of advancements (if not more) before your scenario becomes reality - but even then there are significant physics barriers to cross.
They're focusing on gaming because they're pursuing the consumer market. The things you describe sound like a much better fit for the business market (and there are people, likely including Magic Leap, pursuing it).
But what you describe isn't quite there technologically; they just don't have the resolution or refresh rate and input methods are still evolving. Another issue with AR is that it's only additive. The lenses make everything darker and they can only add light.
Some of the best use-cases I've seen for AR is assistive technology where it walks you through taking something apart labeling each part. Right now that requires an incredible amount of manual resources to generate (or would require a huge investment and significant breakthroughs to make dynamically generated or procedural). That tech is orthogonal to the display and input, but both are necessary for a product.
Check out [SimulaVR](https://github.com/SimulaVR/Simula) which is trying to do EXACTLY THIS on the Linux platform. We're in need of developers and test users if anyone is interested in joining the fight.
I am not sure it's fair to say 'this entire field'. VR is very focused on gaming, but AR has already placed a huge focus on productivity. Hololens and also the Google Glass team (yeah that's still being developed) have placed a lot of focus on business applications, and I believe right now that's the sole focus of glass given it's failure to launch in the consumer space. I think Magic Leap are positioning themselves as the next gen iPhone - they need to get that cool factor to win over the early adopters, and then later will broaden into the productivity space. But from what little information they have released, productivity and non gaming applications is also a large focus of theirs, and I don't think this page indicates otherwise.
If you want to reach HD resolution displays at a "virtual" 50 cm distance (for example) you'd need a VR display resolution at like 10K or something, very high in any case. And even then, would it be more ergonomic than a regular screen? It would only pay off if it was something you could use for hours a day, like you can with regular screens atm.
Same with the gloves, if it doesn't have good feedback then I'm going to nope out. Good for tech demos and sci-fi movies, but in real life you need the feedback and physical object.
(you know, like how in real life I need a keyboard and mouse instead of just an awesome mechancial keyboard and vim / emacs mastery).
Yea, but if the display was a "virtual" 50cm away, it could be a much lower resolution, or a much bigger size. The pixel density doesn't need to depend on the resolution of the virtual screen, it just needs to be good enough so the user can't distinguish the pixels of the VR itself. Then any resolution can be accurately simulated in the VR.
There are lots of companies working on AR/VR for productivity. Microsoft Hololens, All of the Windows Mixed Reality headsets, Meta, DAQRI (not so much for your description of productivity. eg word processing), and ODG. If you want a headset that has the best resolution, you should check out the Pimax (https://www.kickstarter.com/projects/pimax8kvr/pimax-the-wor...). There are mixed reviews on the headset itself, but they have very high resolution (Pimax 8K VR) that's 4K per eye.
4k per eye isn't even close to enough for grandparents wishes... You'd need at least 10k per eye, and would still have a lower resolution than a 24 inch full HD screen at 50 cm
The company I work for is working on a VR productivity app that is focused entirely on using your existing desktop apps in a VR space at speeds similar to what you can do with a keyboard and mouse. We have essentially taken the tech for the orbiTouch[0] and mapped it to the controllers that come with the current headsets.
We are always looking for more alpha testers. Should you have a VR setup feel free to contact me at davidn@blueorb.com to try it.
I fully agree. But remember that gamers are passionate folk who are willing to throw their hard-earned money away at this hobby. That revenue will be used, we hope, to develop the next generation of displays which will have sufficient foveal resolution to be used as a monitor replacement. I would love to leave my dual 27" monitor desk behind and travel the world while doing my work. That day will come. But I'm afraid that it'll be too late for me - I'll have already retired.
VR is mostly focused on gaming. Because it sells better now and is simpler, cheaper. AR is mostly focusing on produtivity, Google Labs and Microsoft Hololens are known examples. But this is harder to develop, so progress is simply slower. And I assume it's easier to develop for highly specific task then delivering a universal solution. So there is not much to hear about for the normal people.
Have you tried Samsung Gear VR with Galaxy S8? S8 has the latest and greatest mobile display with one of the highest pixel density in the market. However, when I use it in a VR headset, I can still see the pixels. It may be good enough for some gaming but it's not usable for working with spreadsheets in VR.
I think the focus is on gaming because gaming purchases are made with disposable income on disposable items. Selling goggles to a gamer just requires convincing the gamer that goggles are cool.
On the other hand, selling into business means developing business solutions and making business cases for purchase of the solution.
On the third hand, your use case is almost certainly smaller than the gamer market and has many of the same business case features as the business market...which means it pretty much is a business case in that "viewing virtual displays in high definition" is a business solution involving Excel and web development and hammocks and trains. Unlike the game market, the developers of the tools you want to use are not chomping at the bit to work with MagicLeap.
I see some potential to AR for assisted training of employees. Imagine being able to train someone to a series of tasks that are displayed right in front of the user's eyes.
We learn much faster by copying someone, and having this kind of tool would free up resources while you on-board someone.
Why, because AR needs to be bright enough to not be drowned out (even with Magic focus cues) by background light. Right now, that means that they project a small field of view. It appears that both MagicLeap and HoloLens are ~35deg FOV. A typical max resolution would be 30-60pix/deg (neither achieves this!). This means that you can get at most a single FHD resolution monitor in view at a time.
If you want to have something like a seemless experience, you need something closer to 90deg field of view and several thousand linear pixels (e.g. 5-10Mpix per eye) at near retina resolution. That is VERY hard even for the biggest GPUs (that don't fit on your belt) even at 60Hz. If you want to avoid dizzyness, then you want >90Hz and <10ms latency.
>I don't understand why this entire field seems so focused on gaming, and not productivity.
Probably for the same reasons that Tesla built expensive sports cars first, before focusing on delivering a (comparatively) more boring commuter sedan.
That's a really good point. I've had friends in the past show my their VR demo with multi screens, and it's been awesome. Of course, resolution is the barrier to it being usable.
There're other interesting areas for productivity improvements too, like better interfaces for human computer interaction
Your post gave me a fun thought: someday, we won't buy desktop and laptop computers anymore. We'll buy hi-res VR goggles and keep our desktop environments in the cloud, using virtual keyboards and hand sensors to interact with them.
The latency issue makes this essentially impossible since anything that's not trivially close to the local network will have upwards of 40ms of latency, making VR mostly useless unless the local compute does significant processing.
Because more people like to play games than work with abstract information, so that's where the money is. I think your notion that the sale of 3bn laptops means 3bn want to work at all hours is a little naive.
I bought a Pimax 8K-X on kickstarter for this. I use my vive and rift but the screen door effect and pixels make it hard to write too much code unless im in the right mood.
Those ideals of productivity sound amazing but in practice I can see problems. A floating, non corporeal keyboard with no feedback would be tough to type on for starters.
They definitely are trying to build what I want. There is even an Excel spreadsheet in the marketing materials! I didn't think the Oculus Rift had good enough resolution though. Has anyone tried this?
>I don't understand why this entire field seems so focused on gaming, and not productivity
Because they think that gamers will pay. Unfortunately, I see that they got a wrong idea. It has a vibe of a semibotched Kickstarter project, except backers here are not private individuals, but gigacompanies.
A type of a gamer who spends 15k usd on a gaming rig to crush opponents in Quake 3 in ultracompetitive environment, will not care a bit about this toy.
The founder of the company comes from a socioeconomic strata whose people have that characteristic. A Boston "old money family (R)" born man may see that selling gaming stuff to quite a lot of relatively rich people dumping 15k on a gaming rig is a good business idea, proceeds to build a company built around that idea with all audicious bold claims being received with accolades from other people like him, but never actually bothers to figure out what things matter in a gaming gear.
If you have read his personal blog from naughties before he deleted it, you will get that his ways can be said to be well beyond "nebulous". He wrote stuff like "solving global problems" while maintaining that tone you usually see from people who flood the internet with something very insubstantial like "saving African children with Agile, innovation, and seven sigma framework..."
Ok, back to the botches kickstarter line. As happens often with such projects, original claims performance get scaled down, company barely manages to deliver a downrated product after missing the delivery deadlines multiple times, product works so so, and in the end it ends in your drawer for good. A year down the line the company simply shuts down the cloud service for the widget and you are left with an expensive paperweight. I expect magic leap to follow this route.
There are two things that would absolutely revolutionize how I work that I would pay big money for: 1) Any sort of glasses that would allow me to view virtual displays in high definition. I don't care if I have to turn my head to see more than one display, I don't care if they are VR instead of AR. Should be high enough resolution to be able to use busy Excel spreadsheets and see enough details on a page to do web development 2) Some sort of glove where I could move my fingers to type. It doesn't need to represent an actual keyboard. I could learn whatever new gestures are required for each character.
Those two innovations would mean freedom for me. You could effectively work anywhere in any position, laying in a hammock, on a crowded train, at night in bed when inspiration hits without waking your significant other. It would have to be AR to use while running or working out :)
If these can be made with enough quality to enable equal productivity to a laptop the creators will have an addressable market of about 3 billion people.