People often forget this part of what Gordon Moore said. Perhaps the final barrier will not be technological, but economic in nature? (Reminds me of issues with access to space.)
I find that hard to swallow. We live in a world where things get cheaper with time. I'm sure the cost in some kind of absolute wealth measure increases very fast. But by this measure, a single one of today's factories would be completely unaffordable to humans a hundred years ago, but today we generate more wealth (in that absolute measure) in one day than we did in a year back then.
I'm sure the cost of these advanced factories is unfathomable to us today, but in X years, in proportion to the wealth available to build them, I believe they will be affordable.
As a final nail in the coffin, this assumes that our manufacturing processes remain more or less the same. What about, for example, nanomanufacturing? The promise of wide-spread nanomanufacturing is that anyone will be able to build cpus in their backyard...
