NovelAI is literally built by disgruntled AI Dungeon fans after the censorship scandal who wanted a better, more privacy-focused AI Dungeon for story generation. If you think AI dungeon has some cool features, you ain't seen nothing yet. There's a prompt tuning system called "modules" which basically primes the AI to write in a certain style, like a certain author or genre, etc (there are dozens of default modules, plus an included API to train your own, and you can share them and import other people's modules freely. Want a module trained on all of Harry Potter, or trained specifically on a particular genre of erotica? Someone's probably already made one). That's only one of the cool features it has, there are plenty others.

General upsides: There's a billion different settings and dials and output customization. The context limit is 2048 tokens is almost 3 times larger than AI Dungeon's. Everything is extremely transparent as far as inputs and outputs go, and the UI is really clean and functional, it's more like a word processor than anything. There's no crappy energy system like AI Dungeon either. Everything is run on private servers and encrypted so the devs couldn't snoop on your stories even if they wanted to (which was the main reason it was created, after it came out the AI Dungeon devs were reading and censoring private stories) and there's no risk of Google or Microsoft butting in and forcing them to change their business model like AI Dungeon. It's even got a legacy DO/SAY/STORY choose-your-own-adventure mode/module for people nostalgic about the AI Dungeon aesthetic. The base AI is finetuned on a large, curated collection of novels, rather than on a bunch of garbage CYOA stories written by 10-year-olds like AI Dungeon.

General downsides: It's paid subscription only, though I think there's actually a free trial nowadays. It uses eleutherAI's GPT-J-6B, which is about 1/40th the size of GPT-3's largest model which is 275B. It definitely punches above its weight class though and subjectively has comparable or higher output quality to AI Dungeon's "Griffin" which used a smaller GPT-3. And they're coming out with a 20B model too in like...a week or two.

Do you know how much you can paste from an existing story in NovelAI (like number of words) to continue off of? Like I've got some 2500 word stories I never got around to writing an ending (I struggle with endings in general), that maybe I could paste into NovelAI and see where it goes from there. Might help me finally finish the damn things, they've been sitting in a folder for a couple of years now.

I don't think there's any practical limit (I could probably paste in the entire text of the Bible and see what it generates next), but for the most part you want to use it as a writing companion, not an auto-writer. The general strategy is to have it generate some text (a few sentences), and then generally edit or retry it. The fun part is seeing the novel (ha) ideas that the AI comes up with and then working from there. There are some people who actually do try to use the AI to write the whole thing with not editing, with limited success (usually it involves a very meticulous and nebulous combination of tricks and techniques, and a lot of retrying).

As far as the context limit (how many lines back it can actually use), I believe it's 2048 tokens for novelAI, which is about 3 times higher than AI Dungeon uses, but likely not enough for the AI to have all of the story in memory. For important details or characters that aren't repeated except at the start, you may need to do some priming techniques with memory, author's note, or world info.

Though with 2500 words one advantage is that you have a very large amount of text in your writing style for the AI to try to mimic. One disadvantage of NovelAI over AI Dungeon is that AI dungeon seems a lot better at working off a small, limited prompt with very little context, while NovelAI prefer much longer prompts with plenty of descriptive detail. Having a large prompt that is entirely your own words definitely helps.

Okay cool. Yeah I didn't mean generate it all for me, just come up with some responses and maybe a few ideas of where it could go. I did end up experimenting with it last night briefly and it did help me write a few more paragraphs to one of my stories.

One thing I did notice is it doesn't seem to know when to move on to something else, and will just keep trying to do what's been working, so I need to be able to tell it "Stop that conversation, let's talk about this now", although admittedly it's from very limited interaction with it. Still pretty neat.