.ogg for Gamesalad on the iPhone/Touch is a Major Mistake
designerX
Member Posts: 8
(long post ahead... you've been warned... you may wanna read it though as I think it's very important)
Hi everyone, I just started using Gamesalad for the past hour and so far I like what I'm seeing; that is until I came into the audio part. So, I imported a .WAV sound effect and was surprised that it converted it to Ogg Vorbis. So before continuing with the engine, I looked for more information on the Gamesalad site and see if this is the only format allowed.
"Supported sound file types (.ogg, .m4a, .mp3, .wav, .aiff, and all other Core Audio formats) can be converted into either .ogg (for sound effects) or .m4a (for music)."
(To the devs and readers alike, I haven't purchased the $99 express membership yet, so if Gamesalad re-converts those sound effects to another format that doesn't incur massive CPU resource problems, like Linear PCM (.WAV), on the actual app package that you upload to the iPhone, please disregard the whole post. Although, if you want to learn the basics of how the iPhone handles audio read on.)
See, I like .ogg, it promotes freedom, but it is also a memory hog, especially on the iPhone. For an engine like Gamesalad which consumes a lot of memory already, the additional toll on the CPU of decoding and decompressing .ogg in addition to .m4a (ALAC) is not just bad, it's unacceptable.
... I'm guessing a good portion of the community might be unaware of the implications of this so I'm going to try and explain as much of it as possible in a handy Q&A format.
Dude: Dude, ".ogg," ".m4a," compressed, uncompressed, you lost me bro, why should I care about these things?
X: I'm glad you asked dude. See, the iPhone supports a couple of formats for single and multiple playback. For multiple playback there are formats that are generally referred to as "Uncompressed," those are Linear PCM (WAV) and IMA4 (IMA/ADPCM) audio files; you can play several of these files simultaneously. They are great for sound effects. They also consume less memory but their file sizes are bigger. For single playback or generally referred to as "Compressed," those are AAC, MP3, and ALAC (M4A) audio files; you can only play them one at a time. They are best suited as background music. Be warned though, they consume more memory, but file sizes are smaller.
-----
Dude: Whoa! I didn't even know about that dude. But you forgot ".ogg" man, why are you so against it?
X: Easy there fella, I'm not against it. Vorbis is a free compressed audio codec that is commonly used with the OGG container format, hence .ogg. So, as you can see, Apple doesn't support it whatsoever. So in order to play that on the iPhone you have to port the toolkit, then do the conversion from OGG to WAV internally on the device. That requires MASSIVE resources, as much as 20% of the CPU on the original iPhone. Add to that Gamesalad's footprint and you are left with an extremely small amount of memory to work on!
-----
Dude: 20%!!! I ate 20 burritos before and that was intense. So it's also compressed, does it mean that MP3s also consume the same amount of memory? Then why should I bother using it?
X: Regarding the memory, the answer to that is no. The iPhone has dedicated hardware for optimized MP3 playback. So yes, it consumes more memory than WAVs but not as much as OGGs because of it. Regarding on why bother using it? Well, if you have several background music files that are quite long and you don't want to go over the 10MB 3G "Over the Air" download limit, you can use it.
-----
Dude: Hah! Little birds told me that id's Wolfenstein 3d used OGG. Why should I follow what you say when a genius like John Carmack used it?
X: I know, I wish I was as smart as him. But you missed the point, Wolfenstein 3d used a proprietary engine that is optimized to do only one thing. That is, to run Wolfenstein 3d, so it still has some memory left to convert OGG. See, Gamesalad is a type of engine that can be used to make a lot of different games, therefore, it tends to have a larger memory footprint as it has to be ready to accommodate a wide variety of genres. Oh yeah, did your birds tell you that they stopped using OGGs and used MP3s when they ported Doom because they need extra memory?
-----
Dude: <Fries the little birds> Man you want some? They're delicious. One second though, why do you keep on saying MP3s? I don't wanna use MP3s dude, I don't wanna pay THE MAN to use it. POWER TO THE PEOPLE!
X: Wow! I'm surprised that you even know that you have to pay royalties to use MP3s on your games, software, etc. I'll be happy to inform you though that if you are already accepted in the iPhone Developer Program you're already licensed to use MP3s on your iPhone apps!
(more info on MP3 licensing here... http://www.mp3licensing.com/royalty/)
-----
Dude: That's Rad! I'm now stoked... wha... wha... what's THIS?! Gamesalad automatically converts my sound effects to OGG and my background music to M4A! What's going on man? Think about the CPU dude! Help me!
X: Yeah... that's kinda what this whole post is all about. I guess Gamesalad doesn't want to use MP3s because they have to pay royalties when they can use OGGs for free. Add to that the option of uploading your game to the site which in turn requires YOU to get a license. That is not that big of a problem though as there is M4A.
The biggest problem with Gamesalad at the moment, which is in beta to be fair, is the forced OGG and M4A without the option of using uncompressed formats. Playing two compressed audio formats at once is just an unnecessary burden to the limited iPhone CPU. Think about the number of unnecessary particles, special effects, and lens flares you can add to your game if you can only access those extra CPU resources!
Dude: Lens flares are AWESOME!
<end>
There are a couple of other, I would say, necessary features that Gamesalad should have, like support for PVRTC, Sprite Sheets (If it has support for Sprite Sheets I apologize, I didn't see it when I was quickly browsing the engine), Social Site Integration (FB, twitter, etc.) and a couple more, but it is in beta for a reason. This post was made to help the community and highlight one potentially huge memory saver for the engine. I didn't mean to offend anybody.
Hi everyone, I just started using Gamesalad for the past hour and so far I like what I'm seeing; that is until I came into the audio part. So, I imported a .WAV sound effect and was surprised that it converted it to Ogg Vorbis. So before continuing with the engine, I looked for more information on the Gamesalad site and see if this is the only format allowed.
"Supported sound file types (.ogg, .m4a, .mp3, .wav, .aiff, and all other Core Audio formats) can be converted into either .ogg (for sound effects) or .m4a (for music)."
(To the devs and readers alike, I haven't purchased the $99 express membership yet, so if Gamesalad re-converts those sound effects to another format that doesn't incur massive CPU resource problems, like Linear PCM (.WAV), on the actual app package that you upload to the iPhone, please disregard the whole post. Although, if you want to learn the basics of how the iPhone handles audio read on.)
See, I like .ogg, it promotes freedom, but it is also a memory hog, especially on the iPhone. For an engine like Gamesalad which consumes a lot of memory already, the additional toll on the CPU of decoding and decompressing .ogg in addition to .m4a (ALAC) is not just bad, it's unacceptable.
... I'm guessing a good portion of the community might be unaware of the implications of this so I'm going to try and explain as much of it as possible in a handy Q&A format.
Dude: Dude, ".ogg," ".m4a," compressed, uncompressed, you lost me bro, why should I care about these things?
X: I'm glad you asked dude. See, the iPhone supports a couple of formats for single and multiple playback. For multiple playback there are formats that are generally referred to as "Uncompressed," those are Linear PCM (WAV) and IMA4 (IMA/ADPCM) audio files; you can play several of these files simultaneously. They are great for sound effects. They also consume less memory but their file sizes are bigger. For single playback or generally referred to as "Compressed," those are AAC, MP3, and ALAC (M4A) audio files; you can only play them one at a time. They are best suited as background music. Be warned though, they consume more memory, but file sizes are smaller.
-----
Dude: Whoa! I didn't even know about that dude. But you forgot ".ogg" man, why are you so against it?
X: Easy there fella, I'm not against it. Vorbis is a free compressed audio codec that is commonly used with the OGG container format, hence .ogg. So, as you can see, Apple doesn't support it whatsoever. So in order to play that on the iPhone you have to port the toolkit, then do the conversion from OGG to WAV internally on the device. That requires MASSIVE resources, as much as 20% of the CPU on the original iPhone. Add to that Gamesalad's footprint and you are left with an extremely small amount of memory to work on!
-----
Dude: 20%!!! I ate 20 burritos before and that was intense. So it's also compressed, does it mean that MP3s also consume the same amount of memory? Then why should I bother using it?
X: Regarding the memory, the answer to that is no. The iPhone has dedicated hardware for optimized MP3 playback. So yes, it consumes more memory than WAVs but not as much as OGGs because of it. Regarding on why bother using it? Well, if you have several background music files that are quite long and you don't want to go over the 10MB 3G "Over the Air" download limit, you can use it.
-----
Dude: Hah! Little birds told me that id's Wolfenstein 3d used OGG. Why should I follow what you say when a genius like John Carmack used it?
X: I know, I wish I was as smart as him. But you missed the point, Wolfenstein 3d used a proprietary engine that is optimized to do only one thing. That is, to run Wolfenstein 3d, so it still has some memory left to convert OGG. See, Gamesalad is a type of engine that can be used to make a lot of different games, therefore, it tends to have a larger memory footprint as it has to be ready to accommodate a wide variety of genres. Oh yeah, did your birds tell you that they stopped using OGGs and used MP3s when they ported Doom because they need extra memory?
-----
Dude: <Fries the little birds> Man you want some? They're delicious. One second though, why do you keep on saying MP3s? I don't wanna use MP3s dude, I don't wanna pay THE MAN to use it. POWER TO THE PEOPLE!
X: Wow! I'm surprised that you even know that you have to pay royalties to use MP3s on your games, software, etc. I'll be happy to inform you though that if you are already accepted in the iPhone Developer Program you're already licensed to use MP3s on your iPhone apps!
(more info on MP3 licensing here... http://www.mp3licensing.com/royalty/)
-----
Dude: That's Rad! I'm now stoked... wha... wha... what's THIS?! Gamesalad automatically converts my sound effects to OGG and my background music to M4A! What's going on man? Think about the CPU dude! Help me!
X: Yeah... that's kinda what this whole post is all about. I guess Gamesalad doesn't want to use MP3s because they have to pay royalties when they can use OGGs for free. Add to that the option of uploading your game to the site which in turn requires YOU to get a license. That is not that big of a problem though as there is M4A.
The biggest problem with Gamesalad at the moment, which is in beta to be fair, is the forced OGG and M4A without the option of using uncompressed formats. Playing two compressed audio formats at once is just an unnecessary burden to the limited iPhone CPU. Think about the number of unnecessary particles, special effects, and lens flares you can add to your game if you can only access those extra CPU resources!
Dude: Lens flares are AWESOME!
<end>
There are a couple of other, I would say, necessary features that Gamesalad should have, like support for PVRTC, Sprite Sheets (If it has support for Sprite Sheets I apologize, I didn't see it when I was quickly browsing the engine), Social Site Integration (FB, twitter, etc.) and a couple more, but it is in beta for a reason. This post was made to help the community and highlight one potentially huge memory saver for the engine. I didn't mean to offend anybody.
Comments
That's not the biggest problem...
But hey, welcome to the forums!
QS
Dr. Sam Beckett never returned home...
Twitter: https://twitter.com/Quantum_Sheep
Web: https://quantumsheep.itch.io
Send and Receive Data using your own Server Tutorial! | Vote for A Long Way Home on Steam Greenlight! | Ten Years Left
DesignerX, welcome to the forum. I can understand your position. And it sounds accurate. However, I will mention that when I added sound effects to my game I noticed 0 drop in frame rate.
Now I don't have 20 different sound going on at once, but I do have maybe 4-5 going off on a pretty consistent basis, sometimes as many as 8-9. Again I have not noticed ANY slowdown. My game is pretty spawn heavy also.
So, just give it a try, I think the GS guys did a pretty good job optimizing it with the latest patch. GS isn't going to be making a GTA game anytime soon, but for the type of games that it is making currently, it's pretty efficient now.
JGary
If the OGG audio is the lowest memory hog, then this engine has no reason to exist quite frankly. I downloaded a lite of your Z for Zombie and it tends to quit quite frequently on the original iPhone, which is my main dev kit.
It is easy too dismiss "Eh, it's just audio," but no. Ogg is not supported in the iPhone, it's a major memory hog, and more likely than not the reason why a lot of Gamesalad apps tend to quit on the older iDevices. Playing Oggs in succession (while playing m4a in the background) with the amount of memory left after all is said and done is suicide.
There several things that can be done to save memory on images like the PVRTC format, Sprite Sheets, reducing the quality of your PNGs (You can hide a lot of imperfection with the iPhone screen), but I might do another post on that and elaborate more.
Little birds have told me that Apple itself was at one time recommending developers use Mono sound files at 11khz in .ogg format.
Those little birds can be a devious bunch though.
On a serious note, I like the way you write and explain things. Again, welcome to the forums. Perhaps you'd consider using the free GS software (without buying a licence) to see how much audio affects performance?
As others here have pointed out, real world experience of using the engine suggests that audio is the least of the problems here. I'm not discounting what you're saying, but personally I've seen more obvious performance hits when spawning a bunch of actors for example.
Cheers,
QS
Oh, and this rubbed me up the wrong way:
"If the OGG audio is the lowest memory hog, then this engine has no reason to exist quite frankly"
This engine has allowed teenage kids, pompous students, blue collar workers, nurses, graphic designers, joe public and pretty much everyone else who has a licence to make and publish an iPhone game.
Some have even made some money from it.
I think it's more than proved it has a reason to exist.
Dr. Sam Beckett never returned home...
Twitter: https://twitter.com/Quantum_Sheep
Web: https://quantumsheep.itch.io
Thanks man. I haven't bought a membership mainly because of this and a few other missing features/design decisions, but so far it seems alright. Maybe the GS team is "re-converting" the sound effects (OGGs) back now to WAVs? If I read it correctly you upload your game to GS and then it will send you the link for your packaged app. Can you try and see if those sound effects files are OGG? The ones for the iPhone
I haven't heard that but the fact is that Apple can't recommend something that they don't support.
Here's the actual Multimedia support guide. http://developer.apple.com/iphone/library/documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/AudioandVideoTechnologies/AudioandVideoTechnologies.html#//apple_ref/doc/uid/TP40007072-CH19-SW9
Oh btw, I'm not referring to the iMac, iPhone Simulator (running on your Mac), or MacBook Pro; I'm solely referring to the actual iPhone/Touch. OGGs won't have any memory issues running when you're testing your app on your desktop or laptop, it's a completely different beast when it's running on the iPhone though.
Anyway, I didn't mean it that "reason to exist" that harshly, it's just a figure of speech, sorry if it offended anyone.
I understand what your saying, but you'll just have to see it in action for yourself. Nothing I say will convince ya otherwise
EDIT: In my above post when I said there was no slowdown that was on the iPhone itself. Just FYI.
Thanks for the reply. Last question is what's your iPhone/Touch model? The reason why I'm so apprehensive to pay right now is I just want to play around with this engine. I already own Unity3d (The best out there for 3D and absolutely fantastic even for 2D), Torque 2D for iPhone (waste of money, pre-ordered it and in it's whole year of existence it's essentially in early Alpha with features listed but not implemented, I haven't touched it since), cocos2d (The best 2D engine out there).
If you use even MP3s for example in Torque, you will see a noticeable dip in framerates, depending on your game on the original iPhone, unless you do something drastic with the source of course.
I'm just intrigued on how quickly you can knock out a prototype with GS and no noticeable dip in atleast a 24 fps game while playing multiple OGGs, trust me, this engine is doing some things right.
I will say that I do not have BG music yet. So I have no idea if that will add any slow down. If you had A LOT of effects going off at once, you may notice slowdown. But for your standard game, you won't notice any. I probably could NOT say that pre-8.0 GS though. I'm not kidding when I say that patch ramped up performance quite a bit.
It all depends on the type of games you wanna make. Wanna make something extremely action heavy? May have to make a few sacrifices. I can deal with that, seeing as to how quick/fun it makes the process.
I've been downloading some of the more resource heavy apps made with GS on the appstore and they tend to crash quite a bit on the oldest iPhone and even on 3G. It unexpectedly quits, which is most definitely a memory problem as you theoretically cannot screw GS up as you don't mess with the code. In fact, the only one that hasn't crashed was "Danger Cats" (Kudos to you, firemaplegames). It did slow down on the original iPhone though (have yet to play it in 3G).
I hope GS doesn't have any sort of memory leak that Torque had in its earlier versions (I think Unity had it at one point also, I'm not sure) where in the memory incrementally gets fuller and fuller the longer you play it; then the app get's slower and slower until the iPhone just terminates the app because it doesn't have resources anymore.
Anyway, thanks for answering my questions Jay. I think I will wait and try to get your app in iTunes when it's done; or any resource heavy GS app made with the current version first and see how it runs. I'm going to stick to Unity or cocos2d for now. Gamesalad is great though, I managed to replicate a stage on one of my simple 2D prototypes in Unity in no time. No graphics of course just those default rectangles.
Since you sound like a dev I'm sure you can understand that not everyone is as memory conscience as others. Also I probably imagine that most of the apps you have tried *might* have been pre .80... When I say FPS doubled I'm not kidding. (at least in my situation)
I'm not sure about the oldest iPhone since I don't test on that, but most of the crashes likely have to due with logic error more than memory error. Testing my game I never have any crashes. But I am VERY thorough in how I do things. It tends to take longer, but the end result is better.
Anyways hopefully you just take the dive at some point. The devs are constantly improving the tool. Cya around.
MP3 is not a free format. you have to pay for it. Indie devs have no money for paying MP3.
"Dude: <Fries the little birds> Man you want some? They're delicious. One second though, why do you keep on saying MP3s? I don't wanna use MP3s dude, I don't wanna pay THE MAN to use it. POWER TO THE PEOPLE!
X: Wow! I'm surprised that you even know that you have to pay royalties to use MP3s on your games, software, etc. I'll be happy to inform you though that if you are already accepted in the iPhone Developer Program you're already licensed to use MP3s on your iPhone apps!
(more info on MP3 licensing here... http://www.mp3licensing.com/royalty/)"
-----
Cheers.
One of my projects becomes unplayable on a first gen ipod touch when the super slinky player ship shoots more than three bullets at a time.
I removed the .ogg file to see if it would make any difference.
Alas, it did not
There are about 100 other things affecting it I'm sure, so not the greatest test ever. But hey, I did give it a quick go
It might be worth further investigation. And the technical opinion of the dev team would be interesting to hear too, I'm sure.
Cheers,
QS
Dr. Sam Beckett never returned home...
Twitter: https://twitter.com/Quantum_Sheep
Web: https://quantumsheep.itch.io
There I can't see anywhere that I am licenced and that page you linked is not about licencing, but about royalties, which is even wors than licencing.
As long as you can't link a page, where it explicitely says, I don't have to pay ANY money to use MP3, your word is as good for me as the word of any politician :-P
If you can prove me, what you say, I am with you. Dude.
And as a side note: I rather advise you to get off the high horse, because with your tone you won't make any friends here...
PS: I don't get it, how quoting, strong, li etc. work in this forum
A interesting little find, at the bottom of the games page
http://www.mp3licensing.com/royalty/games.html
"Note: No license fees are due if less than 5000 copies of a particular game title are distributed"
Keep in mind I don't know what I'm talking about but is that the iPhone converting the sound the first time it's played making that little lag hiccup?
DesignerX is right: decoding the .ogg format is CPU-intensive, especially with the limited resources on the iPhone. For this reason, the GameSalad iPhone engine decompresses your .ogg sounds ahead of time, typically at scene load time.
This way, you get the best of both worlds: the reduced disk usage of compressed sound files, and the playback speed of uncompressed audio. These advantages, however, come at the expense of a little extra time spent loading when a scene starts.
We understand that shorter loading times are a priority for some games, and we are looking at different options for optimizing sound delivery for iPhone and iPod touch.
I'm running multiple audio file simultaneously and 30fps animations and consistently get 42 to 58 FPS according to the GS Viewer app. I'm not really noticing much difference playing natively on my iPhone 3G. Very encouraging.
It's good to note that in the App store you can specify preferred iPod/iPhone version (3G or higher etc.), so as long as the buyer reads that, there will be no surprises.
I am curious about loading all assets as a hidden actor at the beginning of the game to avoid intial stutters. Can that be done with all animations and if so how?
Cheers!
Mike
So basically GS is putting the decompressed audio in a buffer, then passing it to OpenAL. With this, playback will be fine, but load times can vary greatly. An average quality 20 second OGG can add up to 5 seconds of loading time on the original iPhone.
Most SFX are short though, generally 1 to 3 second bleeps, even if you want longer ones, as long as you sample it well (GS can have this feature built-in), those file sizes will be nothing in the grand scheme of things. Also, if you have a fairly complex game and you're going over the 10MB "3G Download Over the Air" limit anyway, it will be better to just use an efficiently sampled WAV regardless of the size as both your Background Music (not to mention, most people prefer to play their own iTunes library anyway) and SFX. More memory for other things and less wait times, it's just better for the player.
I guess my real question is, will the option of uncompressed audio be offered in a future update? (Or just remove OGG support for iPhone projects, and make the engine leaner =P )
I have a couple more questions on other things like PVRTC support, Online High Scores (very important) or Open Feint and the likes integration, Facebook, etc. But I understand that the engine is in Beta. I hope you or the other GS devs will answer this. Thanks, and the engine is looking great.
@Hunnenkoenig
I cannot get off the high horse when I'm not riding it in the first place. This post was meant to help the others in the community in understanding how the device handles audio and get some answers from GS devs on why there isn't an option not to use Ogg Vorbis.
People are naturally skeptics, so I'm not surprised at some of the replies, feel free to Google it if you're curious. Regarding the link, that was just included as a guide because the majority of people are still unaware that using MP3 is not free. MP3 is a non-issue anyway as: a) Gamesalad doesn't allow it; b) there's AAC and ALAC (.m4a) which is also efficiently handled by the device's dedicated hardware.
-peace
If the OGG audio is the lowest memory hog, then this engine has no reason to exist quite frankly. I downloaded a lite of your Z for Zombie and it tends to quit quite frequently on the original iPhone, which is my main dev kit."
That ain't the OGG causing the crashes, Z is for Zombie was created and published on a VERY early version of GS, the first release that supported iphone exports, and it was my first game.
Send and Receive Data using your own Server Tutorial! | Vote for A Long Way Home on Steam Greenlight! | Ten Years Left
I ask, as I think I may add a device type toggle to the menu screen for the user to select (since we cannot currently determine device type in GS) and use the value in that to turn off certain features for performance reasons (i.e. if "1st Gen"/"Better Perf" selected, don't play some sounds, do certain animations, spawn as many of actor X, or what not).