Stardock is smart to do the beta the way they did. If they had of released a full version near the end of programming developing stage it would have been out there on the Pirate sites before the 24th of Aug got here thus cutting into their revenue and profits even more. I've been beta testing for 25 years and I've seen it all. Most beta testers just want a free game and hardly ever send in a bug report. In this case we all paid to be in the beta so really you weren't required to do as a normal beta tester would do. Back in my day we always got a free copy of the finished product for beta testing. I'm finding there aren't many that do that anymore. Today is you pay to be a beta tester either before the game comes out or after it comes out like Paradox (EU and HOI series) and Creative Assembly games (Total War series).

Depends upon how you look at it, are you sure you didn't buy an experience and get a game in return? See an explanation I gave in some of the pre-order? threads that cropped up from time to time:

I agree that putting the buy-in up front is important as a first filter for folks looking for a demo.  You can say it as many times as you want, but the thought that there's an actual pre-build of a game out there gets folks (self-included sometimes) into a Demo mindset.  That was a good choice.  I wouldn't have called it a beta, necessarily, perhaps Alpha, or even just called it something completely different - perhaps "vested customer".

Build turnarounds were good, team communication with customers (through frogboy) was also good.  In other betas or more hierarchic designs, I think it'd be really cool to be able to talk to team-members beyond Frogboy, though I suppose that would expose them to the harshness and timesink that is the forum.

I have mixed feelings about the beta process.

I was expecting the game-play to be more fun from the start and I was expecting to be testing the balance more while still in cloth map mode. At the end of the day if Stardock got what they needed from the process then it has to be called a success - even if the process became a bit frustrating from the testing side.

If what we did was called 'Alpha' instead of 'Beta' then I wouldn't have expected anything more than what we got. It certainly felt more of an alpha test than a beta test because we were mainly focussing on specific mechanics rather than the game itself. Thinking about it, that distinction would have saved me a fair amount of doubt about the final product during testing.

Another element I found frustrating was the lack of specific feedback about ideas in the beta forum. It seemed more feedback was given from the devs in the non-beta forums than in the beta forums and in the end I was posting hoping for some sort of response in other forums rather than providing direct feedback in the beta forum. I'm not sure how the devs processed the suggestions and bugs keeping it separate and controllable. From a beta tester perspective, doing justice to the beta process took an enormous amount of structured time and it felt at times that this time wasn't all that valued. Not that the devs could have done much more than they did - but that was a perception that I had during the process.

Also, there were many suggestions that testers were very passionate about that never got a response back despite obvious frustration and desperation. Even a comment like 'We're unsure at this stage but are discussing it.' would have sufficed at the time.

Having said all that about specific feedback, I was very impressed with the amount of developer information that kept coming through. Unheard of really when you consider their workloads.

As a conclusion I don't know if I would want to go through the process again because of the emotional involvement you get with the project that is hamstrung through the lack of power to change things.

I think that the biggest thing they need to add in the next beta (or alpha, which what it really was) is a way to auto-report bugs. The way to provide dumps was very nonsensical and took a lot of time and effort and required Dropbox. 

If I have a little pop-up that lets me upload the debug.log file and lets me add a little comment, I'd file every bug and crash I could. 

I haven't been in Stardock betas before so I didn't know what was coming. But I agree with other posters, what we tested was an Alpha rather than Beta. The Beta will be the release version, and six months after that the game will be complete.

If that is the way Stardock do things I suppose it is ok. It was fun to be part of the process.

However. For testing to be more successful the builds need to be more stable. If you can't run the game you can't test anything, simple as that. more focus on stability and performance on an early stage is my suggestion for future betas.

And as mentioned above, an automatic bug reporting tool is needed. Or at least a forum with free image and debug-file uploads.

I have been following the beta process from the outside, as when this all began I didn't have the money to plop down a preorder, and by the time I did they had cancelled it (you know, because it was a month from release and all that). So I missed it this time, and I am sad, but after reading over the Elemental BETA forums and the Elemental General forums, I can only say I wish I'd had the money sooner so I could have offered input.

I agree with the fellow who said that a lot of beta testers just sign in for a free demo, never comment, and therefore miss the point entirely. Alternatively, there are a number of people that end up beta testing that should just never be beta testers, because their understanding of things like "balance" and "practicality" and so on are.... negligible.

It's a tough situation, but I suggest the only way to get around that is either to hire professional beta testers or increase the pool of beta testers as best as possible -- the latter being something that the method of "everyone who preorders by X date is a beta tester" does very well.

They were up front with how the beta would be from the start...cloth map only and then would add in features.  Heck it even said on the opening screen about how painful this would be.  I appreciated the honesty and the chance to try it out.  I really loved when they added that old homage to dos games and thought that was great.  They listened to ideas and made changes and have done a great job!