I got dragged into an Agile project a few years back, and expected to hate it. My background is with fixed-price and (semi-)formal-methods projects, so I was wasn't expecting to enjoy the experience.
I was pleasantly surprised: the people I was working with were every bit as disciplined and quality-driven as anyone I'd ever encountered in my consulting career. They were just doing it a completely unfamiliar way.
This year I've been working in another agile organization, and have finally started to figure out why I like it.
My old projects typically used some sort of waterfall process, and we repeatedly bit off more than we could chew. My director went so far as to send every one of the engineers through a week-long project management course, just get get us better at estimating. Our estimating process in those days was so bad, and we were so risk-averse, that the only projects we would take were ones where we badly underestimated their difficulty. Not a good way to chose projects!
With scrum, we do a two-week sprint, then stop and check our progress. That means we find out immediately if we'd underestimated, and can correct it. There are lots of individual sprints, so we can do large projects, but didn't have to get the estimate right before we ever put pen to paper.
The short sprints are easier on the team, too: if we made an error, it was limited in size, so we could recover in later sprints. This was a much friendlier variant of the alternative management process, what the military calls "limited objectives, tight control".
Nobody likes limited objectives, and nobody likes tight control. If you're in that mode, it's because your commander doesn't trust you with anything bigger. It also suggests that someone's told management/command to "Keep it simple, Dave is stupid". I hate when someone says KISS, because I know who the last "S" stands for.
With the old waterfall or limited-objectives processes, even improved by training, we also tended to build the wrong program. Sometimes a very high quality wrong program, but the wrong one. The customers were actively shut out of the process, so we never found our specification errors until the very end of the project, and had to start again.
In the rare cases where we did find a specification problem, it was usually because we couldn't get something to work, We'd drag in a subject matter expert, and they'd tell us "Don't do that, it's the wrong thing. It'll never work, and the customer wouldn't accept it even if it did".
In an agile process, we actually talk regularly with the customers and their director, and they easily spot it when we start to write the wrong program. Customers may not be able to tell if we're writing a particularly good program, but they sure can tell if it's the right one or not.
A different effect of agile helps us with quality, too. Test plans are best written early, from the specifications, and that's what test-directed development does. This also has the pleasant effect of giving a rapid feedback to the specifier if they've asked for something that isn't testable. Untestable requirements are a clear indication of a bad spec, and doing the test plan right after the specification tends to catch that, usually in the first three days of the sprint.
Of course, writing the tests before the code makes for better programs, and has the pleasant side-effect of being easier on the programmer. I like it when I can say "this works, and I can prove it".
Management likes to hear us say that, too, and to know that we can back it up with real evidence. And they love the kind of test coverage we tend to get, which repeatedly comes out far better than before.
With test writing following requirements and developing following test planning, we get a far shorter feedback loop that when we delay test planning and testing until after development. That means that we can feed back changes to the requirements phase of the next sprint, not stop everything and return to the drawing board every time we find a specification bug late in the process.
That's another benefit: if we find something wrong, missing or just plain bogus, we can take it out of the sprint, finish everything else, and then takes a look at the new issue. The whole team isn't standing around while the testers and the specifiers have a bun-fight in the middle of the conference room. Besides, we're all specifiers and testers, so we're not fighting with unfamiliar people from some other silo or stovepipe in the organization.
My managers (and I, when I get dragged into the job) like it better too. A big project where every part needs to work with every other part is insanely hard to manage, and really east to have go over budget. A collection of small steps, one after the other, is way easier and safer.
It's still hard. You have to make sure all the dependencies happen in the right order, but the graph is way less complicated. I don't have to carry around a PERT program written in FORTRAN IV, I can use lightweight (wimpy?) tools like MS Project and still get the right answer.
Customer mangers, too, feel powerful, because they have lots of input into the backlog. Subject to hard dependencies, they have the ability to change the order of tasks in the queue. So they feel more "ownership" of the results, and try hard to give us good feedback. That's the diametrical opposite to some large projects I've had, where the customer managers were having my product stuffed down their throat "will-ye, nill-ye". I hated that.
As a developer, I like producing the right product to a high standard of quality. I like talking with the user community, who know tons of interesting things about their work. That means they can often invent new, off-the-wall ways of doing things, ways that I can program more easily that the traditional boring way.
As a manger, I like not being out of my depth in boiling water, not having to say "keep it simple, stupid" to someone who's smarter that me, and I really really like being on time and budget.
I admit it, I'm anal. I like processes that "just work", and I like quality. I like Agile because it's anal too. If it didn't sound so darned odd, I'd tell people "my methodology is Anal Development!"