Jump to content

MB3 on FreeNAS - high CPU usage during transcoding


hakuna-matata

Recommended Posts

hakuna-matata

Hi everyone,

As the title suggests, my system is experiencing unusually high CPU usage when MB3 transcodes a video file. (ffmpeg)

 

Below is CPU load comparison when the same file is transcoded by MB3 vs Plex:

 

MB3: http://imgur.com/4PP4WIG

Plex: http://imgur.com/KKBVQRZ

 

System info:

Intel® Xeon® CPU E3-1230 V2 @ 3.30GHz

32GB DDR3 1333 ECC RAM

FreeNAS-9.3-STABLE-201501162230.

MediaBrowser 3.0.5490.2 (installed by plugin)

 

Media Info:
Video
CodecH264
ProfileHigh
Level41
Resolution1280x720
Aspect ratio16:9
AnamorphicNo
InterlacedNo
Framerate25
Bitrate2551 kbps
Bit depth8 bit
Pixel formatyuv420p
CABACNo
 
Audio
CodecAAC
Layoutstereo
Sample rate48000 khz
DefaultYes
 
Containermkv

 

Any suggestions? Many thanks in advance.

Note: The transcoding quality preference in MB3 Settings>Playback>Transcoding> is set to Auto.

Edited by hakuna-matata
Link to comment
Share on other sites

thefirstofthe300

It might be that ffmpeg is transcoding the entire file without any throttling occurring. What do the transcode logs look like? Can you post a couple?

 

Sent from my Nexus 7. Pardon my spelling. :)

Link to comment
Share on other sites

hakuna-matata

It might be that ffmpeg is transcoding the entire file without any throttling occurring. What do the transcode logs look like? Can you post a couple?

 

Sent from my Nexus 7. Pardon my spelling. :)

Hi, please find logs attached.

 

Corresponding CPU loads:

Downton: http://imgur.com/4PP4WIG

PatGarrett: http://imgur.com/EKU0Yo4

 

Thanks for looking at this. :)

 

Downton.txt

PatGarrett.txt

Link to comment
Share on other sites

thefirstofthe300

It looks like the transcode isn't being throttled. If you looks at how many frames per second ffmpeg is transcoding, you will see it is about 360. I believe Plex throttles transcoding which is why it takes so little CPU over the length of the movie. Instead MB appears to be grinding through the transcode at full speed in a short period of time.

 

@@Luke Shouldn't transcoding be throttled?

 

Sent from my Nexus 7. Pardon my spelling. :)

Edited by DaBungalow
Link to comment
Share on other sites

  • 1 month later...
thefirstofthe300

Based on what I have seen in the other discussions, throttling the ffmpeg process was disabled because when the alpha releases that had implemented it went beta because low-power machines weren't transcoding fast enough to make it work (a flaw in the server's throttling implementation).

 

Currently, there is a discussion in the General Discussion forum that is making an effort to find a better way to implement transcode throttling. At this point, you will either have to be patient and wait, or you can join in the fray and work to get it done. :)

Link to comment
Share on other sites

  • 2 months later...
stuartsjg

Hello,

 

Im seeing the same thing as many are and for reference ive taken some process screenshots.

 

Background:   Freenas Plex premium user dissatisfied with Plex' handling of Music albums.  Emby recommended to me by colleague. Main reason for setting up media server is ive realised we are spending £50+ per month on subscription music and movies and TV services.   Looking at how much new things we buy or rent , it would be cheaper to go back to buying tracks, albums or movies again and host them myself.   Naturally as with my intention for plex, i would intend to contribute to Emby if i adopt it.

 

System: FreeNAS9.3 on Supermicro X8STi with Xeon W3680, 12Gb ECC Memory.  Movies/TV/music storage: 4 x 2TB Seagate ES drives in RAID-Z1. Everything else, 2 x WD Red 3Tb Mirror.

 

My situation may see up to perhaps 4 users playing media both locally and remote.  With Plex, i was able to play and transcode up to around 8-10 streams from average bit rate sources (eg. 1-2Gb/hr movies and TV).   I was also able to transcode without any issue 2 or 3 steams with the source material 6-7Gb/hr.

 

Setting up Emby was a dream however i was surprised how long it took to scan the library of 26,000 files at 2.2Tb compared to Plex.  [ I was also surprised it couldn't identify for example a movie in folder ..movies\HD - Taken\Taken.mkv  ]  That said, the library response in service is much quicker than Plex so that's the main thing and it can work out music albums so problem solved there.

 

I did then notice this issue of the 2nd, or 3rd, or 4th etc commenced playing video not playing and the 1st, or 2nd etc played video starting to pause after a few minutes.

 

I then found this thing out about the Emby transcode methodology and decided to do some side by side comparisons with Plex.

 

Both were compared with the other media server disabled (plex and emby not being ran together).  Settings were 720p 4Mbps which was a setting applicable to my two test videos which were my own BD rips so have a large file size;

 

Star Trek into Darkness

File size              : 14.0 GiB

Codec ID            : V_MPEG4/ISO/AVC

Overall bit rate    : 15.1 Mbps
Video Bit rate      : 14.1 Mbps
Width                   : 1 920 pixels
Height                 : 800 pixels

 

Interstellar

File size                : 10.0 GiB

Codec ID              : V_MPEG4/ISO/AVC
Overall bit rate      : 9212 Kbps
Video Bit rate        : 8175 Kbps
Width                    : 1 920 pixels
Height                   : 1 080 pixels

 

 

Plex, both streams playing to same PC (Chome full screen), 720p 4Mbps - highest CPU usage - both videos playing with no issues

55579a03dcf99_InterstellarStarTrekDarkne

 

 

Plex, both streams playing to same PC (Chome full screen), 720p 4Mbps - lowest CPU usage - both videos playing with no issues

55579a4c354e1_InterstellarStarTrekDarkne

 

Note on the plex, i was able to throw in a 3rd stream which was a 5.2Mbps 1280x720 MP4 source stream and all 3 played without any noticeable issue.

 

 

Emby, both streams playing to same PC (Chome full screen), 720p 4Mbps high speed setting - highest CPU usage - first playing video (Interstellar) ran OK for few minutes before pausing, the second (Star Trek) playing video barely ran and took about 30 mins to play 10 mins whilst the first played 20 mins in the same time.   The dashboard shown the 1st video as transcoding at about 21fps and never registered anything for the second video.

55579a85054fe_InterstellarStarTrekDarkne

 

 

Emby, One video (Interstellar) playing to PC 720p 4Mbps, transcoding at 60fps in high speed setting.  Perfect playback of this single stream.

55579b3caafef_InterstellarEmby720p4MbpsH

 

Emby, One video (Interstellar) playing to PC 720p 4Mbps, transcoding droped to 40fps a few minutes later in high speed setting.  Perfect playback of this single stream.

55579ba25b9e6_InterstellarEmby720p4MbpsH

 

 

So, not sure if there is anything i can do to fix this except wait on updates which may schedule transcoding more efficiently.   Ive not been using it for long but aside from this, it looks to be a winner over Plex for me.

 

Hope this info is of some use.

Edited by stuartsjg
Link to comment
Share on other sites

Thanks for the feedback! I will take a look into this. As far as longer scan times, we download more metadata by default, so that is largely the reason. In the metadata section of the web interface you can try disabling some images you don't care about, and that should improve the time for future media added.

Link to comment
Share on other sites

stuartsjg

Thanks for the feedback! I will take a look into this. As far as longer scan times, we download more metadata by default, so that is largely the reason. In the metadata section of the web interface you can try disabling some images you don't care about, and that should improve the time for future media added.

 

Hi,  no probs - if i can pull of any other info give me a shout and ill see what i can do.  I still have Plex installed but just disabled for now until i firmly pick one way or another.   As ill just generally be adding a few things a week the scan time is not an issue, i wouldn't hope to have to re-scan the whole library too often so ill leave things set as they are.

 

All the best,

Stuart

Link to comment
Share on other sites

can you grab the emby transcoding logs from those sessions? you can find the path to the log folder displayed on the front page of the server dashboard. thanks.

Link to comment
Share on other sites

stuartsjg

Hello,

 

Ive done the same again and made a set of logs.   i left one stream running the whole time and had the second stream running on two occasions so there are 3 logs.

 

The first stream on its own uses about 30-40% of overall CPU after the initial 80-100% peak and stream two on its own uses 40-50%

 

I did today shut down my windows XP jail (used for online backup and virus scanning) and with plex was able to get two of stream 1 and one of stream 2 running concurrently transcoded to 720p 4Mbps with no jitter or pausing.  CPU was sitting between 90-100% all the time.

 

The log timeline would be:

 

Stream 1

Stream 1 (continued) + Stream 2

Stream 1 (continued)

Stream 1 (continued) + Stream 2 again

Stream 2 solo

Stream 1.txt

Stream 2 again.txt

Stream 2 solo.txt

Stream 2.txt

Link to comment
Share on other sites

Hello,

 

Ive done the same again and made a set of logs.   i left one stream running the whole time and had the second stream running on two occasions so there are 3 logs.

 

The first stream on its own uses about 30-40% of overall CPU after the initial 80-100% peak and stream two on its own uses 40-50%

 

I did today shut down my windows XP jail (used for online backup and virus scanning) and with plex was able to get two of stream 1 and one of stream 2 running concurrently transcoded to 720p 4Mbps with no jitter or pausing.  CPU was sitting between 90-100% all the time.

 

The log timeline would be:

 

Stream 1

Stream 1 (continued) + Stream 2

Stream 1 (continued)

Stream 1 (continued) + Stream 2 again

Stream 2 solo

 

Thanks. For testing purposes, can you go to the server dashboard -> playback -> transcoding. Disable the throttle feature and then repeat. i'd like to compare the results. thanks.

Link to comment
Share on other sites

stuartsjg

Hello,

 

Yip, no probs.  I've attached a few logs here.

 

To try and gauge what CPU usage with each stream and both i slightly changed the order from last time.

 

Stream 1 (file "stream 1a")

Stream 1 (continued, "..1a") + stream 2 (file "stream 2")

Stream 2 (continued, "..2")

Stream 2 (continued, "..2") + stream 1 (file "stream1b")

Stream 1 (continued, "..1b")

 

This should better cover both eventualities with these two test streams running.

 

I've also took a CPU usage screen shot.  You can see the sections up above after 23.00.   The higher section for the first few minutes was likley due to a file transfer which was taking place.  once that stopped, no other process registered more than 2%.

 

I've no doubt a faster CPU would permit these two to play together in emby, unless one hogs the resource.

 

Once again, hope this helps.

Stream 1a, non throttle.txt

Stream 1b, non throttle.txt

Stream 2, non throttle.txt

post-44275-0-62227000-1432074574_thumb.png

Link to comment
Share on other sites

Ok, I wanted to confirm that it wasn't due to a bug in the throttling but is simply due to cpu utilization. i'm working on a change for the next release that will reduce utilization significantly after the initial buffering.

Link to comment
Share on other sites

stuartsjg

Sounds good, looking forward to that.

 

Its a bit off topic but it may be related to having issues using the app on Roku/NowTV box.   If you fast forward past the point which has been transcoded already, it pauses untill the conversion has cought up as far as i can tell.   Haven't seen it else where so may be an app bug.  Only found it today so not fully checked.

Link to comment
Share on other sites

  • 9 months later...
yggdrasil

Hello,

 

Luke: is that last post meant for the Roku/NowTV issue, or the general transcoding problems? Because the latter still persist in my setup (current FreeNAS 9.3, emby 3.0.5821 via FreeNAS plugin on a AsRock c2550d4i w/ 8GB RAM). Like in the OP's case, Plex has no problem transcoding two streams parallel while emby can't keep up with one :(

 

Thank you for this otherwise lovely gem of software.

Link to comment
Share on other sites

Hello,

 

Luke: is that last post meant for the Roku/NowTV issue, or the general transcoding problems? Because the latter still persist in my setup (current FreeNAS 9.3, emby 3.0.5821 via FreeNAS plugin on a AsRock c2550d4i w/ 8GB RAM). Like in the OP's case, Plex has no problem transcoding two streams parallel while emby can't keep up with one :(

 

Thank you for this otherwise lovely gem of software.

 

Hi there, welcome. In order to best help you, please see how to report a problem. Thanks.

Link to comment
Share on other sites

yggdrasil

Hi Luke,

 

soooo... can I take this to mean this issue should have been resolved already and my issues are to be considered new? Well then,

 

Hard- & software:

AsRock C2550D4I

Intel Avoton C2550 Quad-Core

8GB RAM

FreeNAS 9.3-Stable

emby 3.0.5821 via FreeNAS Plugin system

 

When playing a video in your webfrontend without transcoding, everything's fine. As soon I I have it transcoded, the load on the server goes up because ffmpeg tries to use multiple 100% CPU. Since pausing the movie has no effect on the load I'm suspecting the Throttling option has no effect. I could live with that, if the transcoding would actually work right, but the video pauses every few seconds, presumably because ffmpeg can't transcode fast enough.

Switching to Plex Server, having it transcode one or two movies at the same time is no problem, even though the CPU load for the plex transcoder can get even higher sometimes than emby's.

Like I said, it feels basically identically to the OP's problems.

 

Attached is a log from a steaming session.

streaming.log

Link to comment
Share on other sites

You could try disabling transcoding throttling under playback -> transcoding. 

 

Another thing you can do is try chrome where you will see dramatically reduced cpu usage.

Link to comment
Share on other sites

yggdrasil

You could try disabling transcoding throttling under playback -> transcoding. 

 

Another thing you can do is try chrome where you will see dramatically reduced cpu usage.

Playing aroung with the throttling I of course already tried, it seems like a complete noop for me.

 

Ok, I tried chromium, and with it I can stream 2 transcoded movies without problem. How the hell can this be? Given that the backend-server is hardware-wise more than capable of transcoding multiple streams at once (as evidenced by Plex streaming to firefox without problems), why would the frontend make such a huge difference? So is the frontend code the problem? Since I can have multiple transcoded streams in chrome, but if I add a stream in firefox to it it lags (and only the firefox stream! Chromium continues no problem). I would test xbmc as well, but it doesn't seem to have a transcoding option.

Link to comment
Share on other sites

chrome supports the mkv container which gives us a lot of added flexibility. that's why it's the most efficient browser by far, generally with just a video stream copy and no re-encoding.

Link to comment
Share on other sites

yggdrasil

Since I don't now all that much about AV, I have to ponder: how do you mean, no re-encoding? How can I change a videos resolution without reencoding? And how is this tied to mkv, when I can have the same effect using avi containered source material?

Even if Chromium has nice features, my hardware is more than capable of transcoding multiple streams, which should be independent of the frontend used. So imho there still is something up in the backend code handling the transcoding, as hinted by the fact that the "Enable Throttling" setting doesn't seem to do anything for me.

 

Thank you for looking into this.

Link to comment
Share on other sites

If your quality settings require the video stream to be converted, then yes, cpu usage will be high just like firefox. But if you leave the quality on automatic then in many cases we can take the original video stream as-is, package it into mkv on the fly and deliver it to chrome. We can't do that with firefox so that is the explanation for the differences in cpu usage.

 

The throttling works just fine but in order to kick in, the transcoding progress has to jump ahead of your playback progress by a certain amount. If it doesn't then there is no throttling, which is probably what happened for you.

Link to comment
Share on other sites

yggdrasil

If your quality settings require the video stream to be converted, then yes, cpu usage will be high just like firefox. But if you leave the quality on automatic then in many cases we can take the original video stream as-is, package it into mkv on the fly and deliver it to chrome. We can't do that with firefox so that is the explanation for the differences in cpu usage.

 

The throttling works just fine but in order to kick in, the transcoding progress has to jump ahead of your playback progress by a certain amount. If it doesn't then there is no throttling, which is probably what happened for you.

 

ad 1) But I _do_ change the quality, what else could I have meant when saying "transcoding"? Every time I wrote about a video being transcoded in my posts, I specifically went to the menu in the top right and changed the resolution. Yet, with Chromium I can watch streams without problems, unlike firefox. To be absolutely clear: with both browsers I changed the quality multiple times, and every time I had a quality other than automatic (which some videos didn't even offer?!) I had problems in firefox, but not chromium. While the load/CPU-usage on the server was basically the same, and while Plex can serve both browsers with several quality settings just fine. So I still can't see how the container format  passed on to the frontend can influence this.

 

ad 2) Yes, I'm quite aware that throttling itself can only kick in once it's ahead of schedule. What I meant was, the throttling option didn't result in changes regarding the stuttering experience in firefox. After all, it might have been an overzealous throttling algorithm that prevent actual transcoding work being done, but this doesn't seem to be the case.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...