Nikon Q&A: Head of development sees interchangeable-lens slowdown as an aberration, not a red …

The annual CP+ photography tradeshow is now well underway in Japan, and Imaging Resource publisher and editor-in-chief Dave Etchells is right in the thick of it, meeting and interviewing some of the most influential figures in the imaging industry. Kicking off his interviews, Dave met with Tetsuya Yamamoto, General Manager, Development Headquarters at Nikon Corp. Yamamoto-san is the individual within Nikon charged with overall management of their development efforts. As such, he’s the “buck stops here” person with regard to Nikon product planning and development — and this gives great weight to his answers below.

Dave Etchells/Imaging Resource: I’ll start with a bit of a strategy and positioning question — while the D3300 is an incredible package for its retail price of US$650, there are a lot of entry-level interchangeable-lens cameras priced below it, mainly in the mirrorless space. The Nikon 1 line gets down into that price range, but with a smaller sensor than many of its competitors. How do you position the 1-series against its competitors with larger sensors?

Tetsuya Yamamoto/Nikon Corp. (via translator): Compared to other players, mirrorless is different for us in terms of concept. Although the Nikon 1 is the same mirrorless structure, the intent of launching Nikon 1 is that it’s a total system for the current way that people popularly use the camera, which is to — in a very convenient way — carry the camera out with them, take beautiful photos in a casual and convenient way, and share it with other people through social media, like smart phones or smart devices.

We think that since that kind of usage is expanding, Nikon 1 will address that kind of usage especially well. We’d like to continue to launch the Nikon 1 into the market. It’s unfortunate that people like to compare our camera with just the spec of sensor size, because that’s not the only point; we’re talking about the whole system, again, that people can take along with them very casually and in a very convenient way take pictures and share them — and on top of that, it has an interchangeable lens.

Of course, when you look at the stats regarding mirrorless for the whole industry, the numbers are not growing so much in some, it’s a fact. But in particular sectors, like North American females, Nikon 1 is very popular. So in that sense, some of our intent has really addressed the market needs.

DE: We noticed when you launched the Nikon 1 in the U.S., you ran a very extensive marketing campaign. It was interesting — the Nikon 1 itself was very popular, but it seemed like everybody became more aware of mirrorless cameras through that campaign. It wasn’t your intent, maybe, but I think it was very well received.

When we spoke once before, you mentioned yourself that image sharing is going to be a particular focus of Nikon’s going forward, and you’ve had a very robust solution through your WU-1a dongle and also your built-in Wi-Fi in various cameras. We’re still a ways from the ideal scenario, though — the ultimate solution will be for images just to appear on a user’s smart device as soon as they’ve been snapped by the camera, without having to set up a connection or launch an application.

I expect we’ll eventually reach that point, but what do you think the timeframe for that might be, and what do you see as the key obstacles to achieving it?

TY: We cannot talk about the timeframe specifically, unfortunately, but we continue to believe that image sharing is a very important need to address. Of course, under the key concept of “always connect to the internet,” really, are the needs of the target that we’re talking about. It’s for us to design our cameras to allow them to do that easily.

DE: So you can’t speak to specifics or even sort of “crystal ball”?

TY: No, we cannot be so specific to that question. Again, we believe that “always connect” is really the point, and that point is already established in the world of smart phones. So it’s for us to really find a way to co-work with that, or collaborate with those companies.

DE: Although we’re talking about it as an extremely important feature, what do you know about how consumers are actually using Wi-Fi in your cameras to connect to smart devices and online services? Is it one of the features that everyone thinks they have to have, but don’t actually use very much; or do you see evidence that it’s actually used a great deal? What I’m wondering is, sometimes consumers have a checkbox that they have to have, you know, “sensor size” or “Wi-Fi,” but do they actually use it frequently?

TY: It really depends on the users, whether they “check the box” and then use it, or not use it. As Nikon, we’re trying to continue to fulfill the features around this theme; for example, automatic discovery of other devices, which is not something that the users can do themselves.  So there are various things we can do to improve the operability. [Note: Not every camera or user needs Wi-Fi, though.]

Especially for the Df, for an example, we added in some specs, but also took them out, for example. We took out the Wi-Fi and video, too, from the Df;  that was one of our intents, and also strengthened by comments from the pre-launch interviews of consumers.

Having said that, for all of our cameras, we need to continue to think of the fact that people are looking at the final output not on paper, but in digitized formats — in other words, on screens.

[Note: This was very interesting to me; It’s definitely true, but it’s very interesting to hear the person largely guiding development for a camera company with Nikon’s long history explicitly recognize and articulate the change.]

DE: This is switching topics a little bit. We’ve seen a very sudden shift away from low-pass filters over the last year or two. Nikon was the first company to delve into this area with the D800E. How popular has the D800E been compared to the standard D800, and did that influence your decision to move away from low-pass filters across the board?

TY: The original purpose was not to walk away from low-pass filters. For each camera that we launch, we think about how we can most beautifully show the outcome of the photos, and for that purpose, it just so happens that the decision regarding the low-pass filter was to take it out for the D800E and also the D3300. For example, the D3300 has an EXPEED 4 engine, which means that it can help to render the pictures more beautifully, and combined with the image sensor and also the lens performance, we believe that we have the best quality without the low-pass filter. That’s the thinking process.

DE: That’s interesting! Because the D3300 has the EXPEED 4, it can do more processing to better eliminate moiré artifacts, is that correct?

TY: That’s right, moiré reduction has been improved, too, so yes.

DE: This is an area that we at Imaging Resource view with a little bit of dismay — that the consumers are demanding cameras with no low-pass filter. Really, even with the best efforts at anti-moiré processing, we’ve seen moiré and aliasing from cameras made by every manufacturer. Does Nikon think that that’s becoming less of a problem these days? Perhaps due to the very high resolution of sensors, compared to the lenses?

[Note: I’d heard someone say once that the lenses were effectively acting as LPFs in many cases. That was the direction of my question here.]

TY: The lens perspective says that low-pass filters are unnecessary. The root cause of the moiré is not the lens. It’s really about the sensor’s performance, and how finely it can dice up the image. Depending on how you do that, in some locations you can get moiré. As the pixel count goes up, the moiré occurs more in places where we are less disturbed by it, or it’s less eye-catching.

Finding that we can actually improve the total quality of the image because the moiré is in less distracting locations, we’ve decided that we can do without the low-pass filter. As you know, preventing moiré with low-pass filter reduces the tone characteristics around it. The tone characteristics and the lens performance have been improved enough for us to take off the low-pass filter.

[Note: An interesting answer, that areas where moiré will occur shift as sensor resolution rises, and that the places you’re most likely to see it currently are less obtrusive than where it occurred previously. This is certainly true to some extent, but we’ve been surprised to see very large-area swathes of color artifacts in some cases (brick walls at a distance, for instance), and also some very strong luminance aliasing in architectural shots.

Yamamoto san’s point is supported, though, by the strong preference enthusiast users seem to be expressing for cameras without LPFs. The other subtle point here is that an LPF will necessarily reduce the micro-contrast of images; I believe that’s what was meant by “reduces the tone characteristics around it.”]

DE: Switching to a particular camera — while we haven’t completed our analysis of it yet, the D5300 received some very worthwhile improvements in image quality — particularly for video imagery. What can you tell us about what led to those improvements, the nature of them, and how they were accomplished?

TY: The point here, really, in creating this camera, was [effectively] to create a sensor that is small enough to create a video, when we have a huge sensor that can sense a much higher-resolution picture as a still. It’s not a camcorder or a movie camera, so it’s different in that sense — the purpose is for still capture, and the sensor is there to be able to do that.

For example, if you’re talking about a camcorder, 10 megapixels is enough even for 4K, but the D5300 already has 24 megapixels for still images. We have to work on the image processing engine — that’s the part we worked on most of all, to satisfy both movie and still requirements on the D5300. Also, we improved the part where we extract data from the image sensor as well.

DE: I’m curious, then — most DSLRs do video by line skipping. Does the D5300 read out all the lines, and then perform sub-sampling in the processor?

TY: Not to that extent, although the Nikon 1 does.

DE: The Nikon 1 reads out all of the sensor pixels?

TY: Yes.

DE: Ahh… That’s why Nikon 1’s video is very good.

[Note: This was a surprise to me personally; I had thought that the Sony RX10 was the first camera to read out every sensor pixel and then subsample in the processor, but apparently the Nikon 1-series has been doing that from the beginning. The 1-series cameras do in fact produce very nice-looking video, and this full-readout capability is clearly part of that. By way of background, most cameras reduce the resolution of their sensor data to video levels by simply skipping rows of pixels during readout. This is much easier to do, but is why moiré problems are so common in ILC video footage.]

TY: It is a high hurdle to achieve when you’re trying to extract all the data at high speed from a large sensor. Maybe that’s one of the emphases that we should put on Nikon 1.

DE: Yes, probably so, I don’t think people are generally aware of that. It seems like the threshold for that full-resolution video download is one inch, and that APS-C can’t quite get it out. Is the main limitation to reading all the data off an APS-C sensor at video rates just heat generation? Or is it that the circuitry can’t transfer fast enough?

TY: Heat and readout speed, both.  Although we haven’t worked on it yet, we believe that with the proper technology, we’ll be able to resolve this issue.

DE: So you believe APS-C full readout is coming at some point? That’s good news!

Again, this next question is a little bit about positioning and the competition. Nikon was the first company to announce video from a DSLR with the D90 nearly six years ago, at Photokina 2008. Since then, there’s obviously been a lot of competition in that area, notably from archrival Canon, but recently we’re seeing more activity from other players, including Sony and Panasonic — and there’s the just-announced GH4 from Panasonic.

What’s your strategy for competing and positioning Nikon in the video space going forward? (I guess I’m speaking more of the higher-end video market here.)

TY: One of our advantages is the cropping mode, which makes it look as if the focal length is very long. It’s one of our strengths that we have such a multi-pixel canvas — a lot of pixels already, so if you crop it, it doesn’t matter, we still have a lot of pixels left — and that’s good for still and video.  Currently, one way to address the line skipping issue, is to change the viewing angle (effective focal length multiplier), and reduce the line skipping with the crop mode. That’s one way to work around it today.

DE: Oh, I see. And when you’re not skipping pixels, then — well, we’re getting rid of the anti-alias filters in a lot of cases, but then the anti-alias filter would work for video, also.

TY: That’s correct.

[Note: Again, a very interesting point. In crop mode, the video pixels can be the same as the still-image pixels, so the same LPF that would deal with anti-aliasing for still images would work for video imagery as well.]

DE: It seems like video was an area of particular focus for you in developing the D5300. The addition of 1080p60 video, built-in stereo mics, a jack for an external microphone, and I believe it produces clean HDMI output for use with external recorders, is that correct?

TY: Yes.

DE: So all of those things really bring its video capabilities very much into the enthusiast category. Do you see enthusiast videographers as being a particular market that would buy the D5300? Was that part of your intent?

TY:  Yes, it’s really what we aimed for, although we’re never (at this point) going to decide to launch a purely video camera. You can see our intent from the fact that, for example, in the recording format we only have ‘pro’ and nothing else is really decided. So the focus right now is on the clean HDMI output for use [in professional-level applications]. We’re not thinking of supporting the pro resolution that the Hollywood users would like, though, or the true, professional enthusiasts would like, because that would be making a video camera.

DE: And when you say pro resolution, you mean the ProRes codec standard?

TY: Yes.

DE: I think all the way back at CES 2013, you mentioned in an interview with another publication that you had plans for 4K video in the Nikon 1-series. Is that something that might happen in the near term, or is that still some ways off in the future?

TY: We still have a plan, but the timing can’t be disclosed for now. We believe the Nikon 1 is very suited for this kind of usage, though. One of the key points is that we’re waiting for the user infrastructure to be established, to be able to accept that output. If — or when — we launch this, this camera has the necessary performance, because it is reading in full resolution already. When do you think the infrastructure will be ready in the United States, for example?

DE: [chuckles] A long time, yeah. That was actually one of my next questions — stepping back from any specific model, what can you say about where 4K video fits in Nikon’s plans? It looks like the consumer electronics industry is hoping that 4K will be what 3D failed to be; which is, namely, a reason for people to upgrade their home TVs. At Imaging Resource, we’re thinking this is unlikely to occur, unless 4K TVs get very cheap. To us, 4K looks more like a pro level product, used by filmmakers rather than consumers. Where do you see 4K capability fitting into your line?

TY: I personally agree with your opinion.

DE: That it’s more pro-oriented.

TY: I wasn’t really a fan of 3D anyway. I think the commonality between 3D and 4K/8K is the [rendering of] reality, although the meaning of reality is completely different between 3D and 4K and 8K. It has a sense of reality in a totally different way that probably professionals can use. I agree with you that, you know, the question of whether we’re going to have 4K or 8Ks in homes, is really an issue.

DE: Yes, my feeling is, our eyes only have so much resolution, and if you’re sitting across the living room —

TY: Yes, the distance.

DE: Yes. In our house, we have a medium-large living room, so we just upgraded to a 55” HD, but we needed that because the 45” or whatever we had before, our eyes weren’t able to see everything on the screen, even at 1080. So, with 4K — you’d have to have a 100 or 150 inch screen to see all the detail at that viewing distance.

TY: My house is not that big, so we don’t need that.

DE: Ha-ha! Well, your eyes are better than mine, too, perhaps!

We talked about this once in the past, but I wanted to revisit it or elaborate slightly. Many people are trying on-chip phase detect auto-focus, but that’s an area where your 1-series cameras have done very, very well. As I understand it, the reason that the 1-series is so fast-focusing is because it uses phase detect for all of the focusing, not just to approximate and get close. But other manufacturers have told us, well, it’s not accurate enough — on-chip PDAF is not accurate enough to focus all the way — so they switch and do contrast detect for the last little bit. What is it about your auto-focus system that lets you feel you can get the accuracy from phase detect, such that you don’t need to switch to contrast detect?

TY: That’s because we have a hybrid AF — it’s both PDAF and contrast AF.

DE: Oh, so you use both.

TY: Yes, contrast AF, especially for low light situations. [Note: In previous conversations, I learned that the 1 Series cameras switch to contrast AF at low light levels, but use PDAF to do all the focusing under brighter conditions. In that earlier conversation, Nikon said that some competitors don’t make this distinction based on light levels, so always include a contrast detect cycle in their AF process, slowing the focusing speed even when they’re plenty of light available.] And by the way, we believe that PDAF is actually enough, really, to meet the need in most cases, which makes us very confident in the speed of focus tracking.

The real reason why we still keep contrast AF is for video use, because when it comes to movies, it’s not like you need very fast focusing – “faster is better” isn’t really true in movies. It’s not like you focus first, and then you snap the photo — you’re on already on and checking focus, so sometimes, if it’s too fast, it’s actually not optimal.

DE: But with movies and contrast detect, though, if it hunts, you can see that very visibly. Do you also still use phase-detect some in movies?

TY: Yes, you’re right.

DE: You do use phase detect as well.

TY: Yes.

DE: I want to ask about PDAF on large sensors, and as I see it, there are maybe two issues. One is being able to clock the PDAF data off fast enough; but then when we talked once before, you mentioned an obstacle of bringing PDAF to your large sensor cameras is adapting it to work with different lenses that have significantly different focal lengths. I’ll separate those questions back out again. Is there an issue with clocking the data off quickly enough for large sensors with PDAF?

TY: It is a challenge, clocking the data off, compared to the small size format — the CX format — on Nikon 1.

DE: And you had mentioned once before, I think, that it’s an issue adapting the PDAF to work with different focal lengths of lenses? Can you explain that simply? That’s a point I don’t understand.

TY: First of all, for our DSLR with interchangeable lens, with large size sensors, they accept the light coming in from various angles with different lenses. It’s very difficult to detect the phase differential when the angle of the arrays are all different.

[Note: This is very interesting, and maybe a hint at something deeper within on-chip PDAF technology. The translator in this session was exceptionally good, so I think her translation as “angle of the arrays” is accurate. If so, it sounds to me like Yamamoto-san is talking about how the individual phase-detect pixels are structured on the chip’s surface. They’re created to only see light coming from one side of the frame or the other, but there’s a question of how *much* of the light cone they should accept. I believe that a typical wide-angle lens at large aperture is going to have a very wide light cone, while the light exiting telephoto lenses will have a much narrower range of angles of incidence. So a PD array optimized for use with a wide angle lens would perform poorly with a telephoto one, and vice versa.]

DE: I think I understand. So with the small-sensor Nikon 1 and a very short back focus distance (or flange distance), you have a wide angle of incidence, but in a conventional SLR and at a long focal length, you have a very narrow angle, so it’s harder to detect the phase difference. Thank you, I understand — that’s a good answer.

TY: We can correct some for that with our image processing engine. So it’s not impossible..

DE: Ah, not impossible, but difficult.

For my last question, I’ll ask you to look into your crystal ball — Nikon makes very good glass, so you should have a good crystal ball, perhaps. [laughter] After holding up very well for many years while compact camera sales have fallen, sales of interchangeable-lens cameras dropped significantly just this last year. Do you view that as a longer-term trend, or do you think it’s a temporary blip caused by the global economy?

TY: The story answer will be different for compact and ILC. For compact camera, all the other companies say as well that the market is shrinking, because there are alternatives like smart phones.

DE: Yes.

TY: For ILC, we believe it’s more like a blip or a temporary problem, mainly because of China — where we felt we’d have more growth — isn’t growing because of its macro economy issues. This means it still has potential in the market. ILC, including Nikon 1, will improve later on after this temporary situation.

DE: That’s very interesting. So in China, you were projecting growth there, but in their macro economy, growth has slowed down, and there are other problems. Very interesting.

TY: I believe the same, too — the answer is different for the two. Technically, I’d like to answer for compact cameras, there are the advantages to smart phones, because they’re connected to social media, and that’s the way people use them. So when it comes to that, smart phones can be a very good alternative. But the SLRs are different, because even smart phone users buy ILCs to step up their photo creativity and options.

If Nikon successfully makes the case that we have cameras that will allow you to do that well, then people will step up to our cameras. We don’t believe that the ILC market will shrink at all, because the shift that is happening right now in regards to the smart phones is that people who love watching or enjoying photos are now starting to take photos; when they want to take better photos, they will get something other than their smart phones.

DE: Yes, the smart phones are drawing more people into photography than ever before.

TY: Yes.

DE: Well, I see we’re out of time, I want to thank you for taking this much time out of your busy schedule to meet with me. (Our readers thank you, too.)

TY: Thank you very much.

[Note: We ended up having a bit less time than anticipated, due to an NDA product briefing taking up part of the meeting time. As a result, I ended up skipping around in my list of questions quite a bit, and by the time I realized we were about out of time, I didn’t have an opportunity to go back to our reader questions. Profuse apologies for that — I’ll perhaps move reader questions up to the head of the queue when I’m faced with tight interview schedules in the future.]