Firefly开源社区

标题: AAC ADTS音频解码之路 [打印本页]

作者: jingjin221 时间: 2016-3-15 10:56
标题: AAC ADTS音频解码之路
本帖最后由 jingjin221 于 2016-3-16 11:51 编辑

我这里有一份从TS流中解复用过后的AAC码流，用OMXCODEC来解码始终解码不出来，请大家帮忙验证一下,红色部分为错误打印信息E/OMXCodec(6058): [omx.google.aac.decoder] ERROR(0x80001001, 8195)

经过调试发现，是在SoftAAC2::onQueueFilled aacDecoder_ConfigRaw 出问题了导致如上问题！
参考 http://blog.sina.com.cn/s/blog_645b74b90101e9br.html 这个帖子得知在向AAC解码器送真实数据前会调用aacDecoder_ConfigRaw 去重新获取aac的一些信息，如sampleRate和numChannels，保存在CStreamInfo结构体中。而我是直接送的音频数据包括ADTS头部
但是要送的这个起始数据究竟是如何得到的呢？我尝试播放M4A的歌曲来查看这个起始数据，发现起始数据一般是2个字节，但是具体内容就不同了！求大神分析！我尝试自己伪造起始数据，aacDecoder_ConfigRaw通过了，但是解码的时候还是失败了啊！！！

test.rar

434.99 KB, 下载次数: 15, 下载积分: 灯泡 -1 , 经验 -1

AAC文件

作者: jingjin221 时间: 2016-3-16 11:46
本帖最后由 jingjin221 于 2016-3-16 11:53 编辑

经过半天的调试，AAC解码已经正常结贴了吧！做如下总结，也可以希望以后的同学们少走弯路
1.我起初在OMXCODEC下调试，MP3解码没有问题，AAC始终不行，调试发现是由于少送了CSD信息，另外需要设置关键配置meta->setInt32(kKeyIsADTS, 1);于是自己根据ADTS HEADER构造了CSD信息，但是送入解码器总是出现奇奇怪怪的问题。实在无法解释，于是放弃OMXCODEC，利用ACODEC来解码
2.利用ACODEC解码在JAVA层也就是MEDIACODEC，需要配置信息如下
[mw_shl_code=c,false]       if(audio_cfg.stream_type == AUDIO_MP3)
      {
            mimetype = MEDIA_MIMETYPE_AUDIO_MPEG;
            format->setString("mime", mimetype);
            s->mSampleRate = audio_cfg.sampling_frequency;
      }
      else if(audio_cfg.stream_type == AUDIO_AAC_ADTS)
      {
            mimetype = MEDIA_MIMETYPE_AUDIO_AAC;
            format->setString("mime", mimetype);
            s->mSampleRate = audio_cfg.sampling_frequency/2;
            format->setInt32("is-adts", 1);
            format->setInt32("aac-profile", 0x0002);
      }


      channel_configuration = audio_cfg.channel_configuration;

      format->setInt32("sample-rate", s->mSampleRate);
      format->setInt32("channel-count", channel_configuration);[/mw_shl_code]

当然还需自己构造CSD头部，参考如下ADTS解析函数
[mw_shl_code=c,false]static int AAC_ADTS_header_parse(uint8_t *p_data)
{
uint8_t mpeg_version, layer, profile, sampling_frequency_index, channel_configuration;
uint16_t frame_length;

if(!((p_data[0] == 0xFF) && ((p_data[1] & 0xF0) == 0xF0)))    //ADTS syncword all 12 bits must be 1
      return -1;
para.audio.stream_type = AUDIO_AAC_ADTS;

/*
0 for MPEG-4
1 for MPEG-2
*/
mpeg_version = (p_data[1]&0x08) >> 3;
printf("mpeg_version is %d\n", mpeg_version);

/*
0: Main profile
1: Low Complexity profile (LC)
2: Scalable Sampling Rate profile (SSR)
3: (reserved)
*/
profile = (p_data[2]&0xC0)>>6;
printf("profile is %d\n", profile);
para.audio.profile = profile;

/*
There are 13 supported frequencies:
0: 96000 Hz
1: 88200 Hz
2: 64000 Hz
3: 48000 Hz
4: 44100 Hz
5: 32000 Hz
6: 24000 Hz
7: 22050 Hz
8: 16000 Hz
9: 12000 Hz
10: 11025 Hz
11: 8000 Hz
12: 7350 Hz
13: Reserved
14: Reserved
15: frequency is written explictly
*/
unsigned int Sampling_Frequencies[13] = {96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, 7350};
sampling_frequency_index = (p_data[2]&0x3C) >> 2; //check the table Sampling Frequencies
printf("sampling_frequency_index is %d\n", sampling_frequency_index);
para.audio.sampling_frequency = Sampling_Frequencies[sampling_frequency_index];

/**
0: Defined in AOT Specifc Config
1: 1 channel: front-center
2: 2 channels: front-left, front-right
3: 3 channels: front-center, front-left, front-right
4: 4 channels: front-center, front-left, front-right, back-center
5: 5 channels: front-center, front-left, front-right, back-left, back-right
6: 6 channels: front-center, front-left, front-right, back-left, back-right, LFE-channel
7: 8 channels: front-center, front-left, front-right, side-left, side-right, back-left, back-right, LFE-channel
8-15: Reserved
**/
channel_configuration = ((p_data[2]&0x01) << 1) | ((p_data[3]&0xC0)>>6);
printf("channel_configuration is %d\n", channel_configuration);
para.audio.channel_configuration = channel_configuration;

para.audio.csd[0] = (profile << 4) | (sampling_frequency_index >> 1);
para.audio.csd[1] = ((sampling_frequency_index & 0x01) << 7) | (channel_configuration << 3);
printf("csd-0[0x%x][0x%x]\n", para.audio.csd[0], para.audio.csd[1]);

return 0;
}[/mw_shl_code]
3.不知道为什么在AAC下，明明解析出来的采样率是48K，送进解码器出来的声音明显频率高了，我只要在初始化的时候进行了分频操作！
4.ACODEC比OMXCODEC靠谱多了，也许是该死是READ造成的吧！5.在送数据的时候其实并不用剥去ADTS头部
6.靠谱的ANDROID论坛还是stackoverflow.com.国内怎么就出不了这种网站呢？

作者: jingjin221 时间: 2016-3-16 11:49
再附上音频解码的核心代码吧！[mw_shl_code=c,false]static int audio_decoder_init(StagefrightContext *s, struct audio_config audio_cfg){
#ifdef AUDIO_DECODER
//#define PCM_FILE_PLAY_DEBUG
#ifdef PCM_FILE_PLAY_DEBUG
audio_pcm_play(s);
return 0;
#endif

int ret = 0;
sp<MetaData> meta;
const char* mimetype;
int32_t channel_configuration;

printf("%s_%d\n", __FUNCTION__,  __LINE__);
#if (defined OMXCODEC)
meta = new MetaData;
if (meta == NULL) {
printf("cannot allocate MetaData");
return -1;
}

if(audio_cfg.stream_type == AUDIO_MP3)
{
mimetype = MEDIA_MIMETYPE_AUDIO_MPEG;
meta->setCString(kKeyMIMEType, mimetype);
}
else if(audio_cfg.stream_type == AUDIO_AAC_ADTS)
{
mimetype = MEDIA_MIMETYPE_AUDIO_AAC;
meta->setCString(kKeyMIMEType, mimetype);
meta->setInt32(kKeyIsADTS, 1);
meta->setInt32(kKeyAACProfile, 0x0002);
}

s->mSampleRate = audio_cfg.sampling_frequency;
channel_configuration = audio_cfg.channel_configuration;

meta->setInt32(kKeySampleRate, s->mSampleRate);
meta->setInt32(kKeyChannelCount, channel_configuration);

s->mAudioSource = new sp<MediaSource>();
*s->mAudioSource  = new CStageFrightAudioSource(s, meta);

if (s->mAudioSource == NULL) {
s->mAudioSource = NULL;
printf("Cannot obtain source / mClient");
return -1;
}

if (s->mClient.connect() !=  OK) {
printf("Cannot connect OMX mClient\n");
ret = -1;
goto fail;
}

s->mAudioDecoder= new sp<MediaSource>();
printf("[%s]@OMXCodec::Create____________________________START\n", __FUNCTION__);
*s->mAudioDecoder = OMXCodec::Create(s->mClient.interface(),
meta,
false,
*s->mAudioSource,
NULL,
OMXCodec::kSoftwareCodecsOnly,
NULL);
if (!(s->mAudioDecoder != NULL && (*s->mAudioDecoder)->start() ==  OK)) {
printf("[%s]@Cannot start decoder\n", __FUNCTION__);
ret = -1;
s->mClient.disconnect();
s->mAudioSource = NULL;
s->mAudioDecoder = NULL;
goto fail;
}
printf("[%s]@OMXCodec::Create____________________________END\n", __FUNCTION__);
fail:
return ret;

#elif (defined ACODEC)
sp<AMessage> format;
format = new AMessage;
if(format == NULL) {
printf("cannot allocate format\n");
return -1;
}

if(audio_cfg.stream_type == AUDIO_MP3)
{
mimetype = MEDIA_MIMETYPE_AUDIO_MPEG;
format->setString("mime", mimetype);
s->mSampleRate = audio_cfg.sampling_frequency;
}
else if(audio_cfg.stream_type == AUDIO_AAC_ADTS)
{
mimetype = MEDIA_MIMETYPE_AUDIO_AAC;
format->setString("mime", mimetype);
s->mSampleRate = audio_cfg.sampling_frequency/2;
format->setInt32("is-adts", 1);
format->setInt32("aac-profile", 0x0002);
}

channel_configuration = audio_cfg.channel_configuration;

format->setInt32("sample-rate", s->mSampleRate);
format->setInt32("channel-count", channel_configuration);

printf("[%s]@ACodec::Create____________________________START\n", __FUNCTION__);

sp<ALooper> mLooper = new ALooper;
mLooper->setName("MediaCodec_Adio_looper");
mLooper->start(
         false,    // runOnCallingThread
         false,    // canCallJava
         PRIORITY_FOREGROUND);
s->mACodecAudioDecoder = MediaCodec::CreateByType(
            mLooper, mimetype, false /* encoder */);
if(s->mACodecAudioDecoder == NULL)
{
printf("Failed to create mACodecAudioDecoder\n");
return -1;
}

ret = s->mACodecAudioDecoder->configure(
            format, NULL /* surface */,
            NULL /* crypto */,
            0 /* flags */);
if(ret != OK)
{
printf("Failed to configure mACodecAudioDecoder\n");
return -1;
}

printf("[%s]@ACodec::Create____________________________END\n", __FUNCTION__);

ret  = s->mACodecAudioDecoder->start();
if(ret != OK)
{
printf("Failed to start mACodecAudioDecoder\n");
return -1;
}

ret = s->mACodecAudioDecoder->getInputBuffers(&s->mAudioInBuffers);
if(ret != OK)
{
printf("Failed to getInputBuffers mACodecAudioDecoder\n");
return -1;
}

ret = s->mACodecAudioDecoder->getOutputBuffers(&s->mAudioOutBuffers);
if(ret != OK)
{
printf("Failed to getOutputBuffers mACodecAudioDecoder\n");
return -1;
}

printf("got %d input and %d output buffers", s->mAudioInBuffers.size(), s->mAudioOutBuffers.size());
fail:
return ret;

#endif

#endif
}
[/mw_shl_code]
[mw_shl_code=c,false]static void* audio_decode_sound_thread(void *arg)
{
status_t err;
StagefrightContext *s = (StagefrightContext*)arg;
MediaBuffer *buffer = NULL;
printf("[%s]Thread id:%d/n", __FUNCTION__, gettid());

if(audio_decoder_init(s, para.audio) == -1)
return NULL;

size_t frameCount = 0;
if (AudioTrack::getMinFrameCount(&frameCount, AUDIO_STREAM_DEFAULT, s->mSampleRate) != NO_ERROR) {
      return NULL;
}

int nbChannels = 2;
int audioFormat = ENCODING_PCM_16BIT;
size_t size =  frameCount * nbChannels * (audioFormat == ENCODING_PCM_16BIT ? 2 : 1);
printf("size is %d, s->mSampleRate is %d\n", size, s->mSampleRate);

s->mAudioTrack = new AudioTrack(AUDIO_STREAM_MUSIC,
s->mSampleRate,
AUDIO_FORMAT_PCM_16_BIT,
AUDIO_CHANNEL_OUT_STEREO,
0,
AUDIO_OUTPUT_FLAG_NONE,
NULL,
NULL,
0);
if ((err = s->mAudioTrack->initCheck()) != OK) {
printf("AudioTrack initCheck failed\n");
s->mAudioTrack.clear();
}
s->mAudioTrack->setVolume(1.0f);
s->mAudioTrack->start();
#if 1
#if (defined OMXCODEC)
while(1)
{
status_t status = (*s->mAudioDecoder)->read(&buffer, NULL);

if (status == OK) {
printf("%s@AUDIO DECODER OK\n", __FUNCTION__);
if (buffer->range_length() == 0)
{
printf("%s:ERROR_BUFFER_TOO_SMALL\n", __FUNCTION__);
status = ERROR_BUFFER_TOO_SMALL;
buffer->release();
buffer = NULL;
continue;
}
//printf("BUFFER RANGE LENGTH[%d]\n", buffer->range_length());
}
else
;//printf("%s@AUDIO DECODER NOT OK\n", __FUNCTION__);

if(status == OK) {
sp<MetaData> outFormat = (*s->mAudioDecoder)->getFormat();
outFormat->findInt32(kKeySampleRate, &s->mSampleRate);
printf("SAMPLERATE[%d]\n", s->mSampleRate);
}

if (status == OK) {
s->mAudioTrack->write(buffer->data(), buffer->range_length());
buffer->release();
buffer = NULL;
}

}
#elif (defined ACODEC)
static int first_flag = true;
int sampleSize;
static int64_t kTimeout_audio = 10000;
size_t inIndex;
size_t outIndex;
size_t offset;
size_t len;
int64_t presentationTimeUs;
uint32_t flags;

while(1)
{
err = s->mACodecAudioDecoder->dequeueInputBuffer(&inIndex, kTimeout_audio);
if (err == OK) {
//printf("filling input buffer %d\n", inIndex);

const sp<ABuffer> &buffer = s->mAudioInBuffers.itemAt(inIndex);

if((para.audio.stream_type == AUDIO_AAC_ADTS) && first_flag)
{
memcpy((uint8_t *)buffer->data(), para.audio.csd, 2);
sampleSize = 2;
first_flag = false;
}
else
{
sampleSize = audio_read_one_frame((uint8_t *)buffer->data());
}
presentationTimeUs = 0;
if(sampleSize <= 0)
break;

if (buffer->capacity() < sampleSize) {
printf("buffer capacity overflow\n");
break;
}

buffer->setRange(0, sampleSize);

err = s->mACodecAudioDecoder->queueInputBuffer(
inIndex,
0 /* offset */,
buffer->size(),
presentationTimeUs,
0 /* flag*/);
//printf("queueInputBuffer err is %d\n", err);
}

err = s->mACodecAudioDecoder->dequeueOutputBuffer(&outIndex, &offset, &len, &presentationTimeUs, &flags, kTimeout_audio);
//printf("dequeueOutputBuffer err is %d\n", err);
if (err == OK) {
s->mACodecAudioDecoder->getOutputBuffers(&s->mAudioOutBuffers);
//printf("got %d output buffers", s->mAudioOutBuffers.size());
const sp<ABuffer> &buffer = s->mAudioOutBuffers.itemAt(outIndex);
//printf("output buffers[%d] size[%d]\n",outIndex, buffer->size());
s->mAudioTrack->write(buffer->data(), buffer->size());
s->mACodecAudioDecoder->releaseOutputBuffer(outIndex);
}
}
#endif
#endif
}[/mw_shl_code]

作者: 牛头 时间: 2016-3-17 09:49
技术贴，顶起来~

作者: 2273431063 时间: 2016-4-11 20:18
你好，我也遇到同样的问题，可以请教一下吗。我的QQ：2273431063

作者: jingjin221 时间: 2016-4-19 17:26

2273431063 发表于 2016-4-11 20:18
你好，我也遇到同样的问题，可以请教一下吗。我的QQ：2273431063

加我QQ512975979

作者: dewettie 时间: 2016-4-22 11:28
你好
你能告诉我哪些文件萤火虫SDK改变 - https://bitbucket.org/T-Firefly/firenow-lollipop/src修复AAC ADTS？
然后，可以在标记红色部分改变或增加

Hi
Can you tell me which files to change in Firefly SDK - https://bitbucket.org/T-Firefly/firenow-lollipop/src to fix AAC ADTS?
Then can mark parts in red to change or add.

作者: jingjin221 时间: 2016-4-22 11:52

dewettie 发表于 2016-4-22 11:28
你好
你能告诉我哪些文件萤火虫SDK改变 - https://bitbucket.org/T-Firefly/firenow-lollipop/src修复AAC ...

你的软硬件平台是什么？

作者: dewettie 时间: 2016-4-23 10:05

jingjin221 发表于 2016-4-22 11:52
你的软硬件平台是什么？

何晶晶221

萤火虫RK3288 - 棒棒堂5.1 - Firenow SDK
HPH RK3288电视盒 - 棒棒堂5.1 - Firenow SDK
下面是我使用SDK的源代码的框架文件夹 - https://bitbucket.org/T-Firefly/ ... ?at=Firefly-RK3288.请告诉哪些文件要修改或添加为您修复。

Firefly RK3288 - Lollipop 5.1 - Firenow SDK
HPH RK3288 TV Box - Lollipop 5.1 - Firenow SDK
Here is the frameworks folder of the source code of the SDK I use - https://bitbucket.org/T-Firefly/ ... ?at=Firefly-RK3288. Please tell which files to modify or add for your fix.

作者: yl586526 时间: 2016-4-26 11:08
支持

作者: dewettie 时间: 2016-4-27 01:11
能否请您分享萤火虫RK3288平台棒棒糖ADTS AAC贴片/变化？
Can you please share ADTS AAC patch/changes for Firefly RK3288 Lollipop platform?

作者: jingjin221 时间: 2016-4-28 09:35

dewettie 发表于 2016-4-27 01:11
能否请您分享萤火虫RK3288平台棒棒糖ADTS AAC贴片/变化？
Can you please share ADTS AAC patch/changes f ...

AV FRAMEWORK 我没有更改，参考的rk3288 android4.4的SDK。
本贴只是描述了我在调用AAC解码时所遇到的一些问题！

欢迎光临 Firefly开源社区 (https://dev.t-firefly.com/)