Jump to content

Many live sources sent through 1 broadcast server need to be archived when broadcast


Recommended Posts

Current Scenario:

I'm working with a internet radio business that sets up encoders at rural radio stations in the US. The rural stations take simple line-level audio from their boards and plug them into a workstation nearby. The workstation is using Windows Media Encoder 9 (I know, I know...) to send a WMA 8 stream back to a Windows Media Services 9.6 server.

 

The rural stations only record very specific shows, so it's not like a 24/7 stream. The streams last for about 2 to 4 hours. Each radio station might have a few broadcasts each week.

 

On the WMS Server there are about 250+ total publishing points, one for each radio station. Each station has its own publishing point. Listeners go to oursitename.com/call-letters to get to the stream for their radio station. Here comes the most important part: Each stream has the "WMS Archive Data Writer" plugin enabled and saves the live streams to a file.

 

Listeners can then retrieve archived broadcasts in case they missed the live show. This is key to our service.

 

 

The Problem:

 

The whole Windows Media Encoder / Expression Encoder / WMS ecosystem can't stream the audio as an MP3. I've looked into Smooth Streaming, HLS, etc. and blah and it has wasted more time than I want to admit. We want to stream to as many devices as possible, namely mobile devices. Thus, we'd like to take the source audio, encode it as an MP3 stream, send it to our streaming server and broadcast it from there.

 

I've looked into IceCast and SHOUTcast, but I'm having a hard time figuring out how to reliably record the source streams when they come in. It appears that some kind of ghetto wget script is the only solution. In essence, running a script every few minutes and checking for incoming streams.

 

The only other option I saw, specific to icecast, was to use the on-connect function of the config file to kick off a wget script of some kind.

 

 

The Question:

Does anyone have suggestions as to what a good product to do this would be? I've looked into IceCast and SHOUTcast but both seem to have rather poor archiving features. And by poor I mean "hack your own wget script together." There are plenty of stream rippers out there, but those are client apps that don't seem to support multiple simultaneous streams and would also need to be scripted to check for live streams every so often.

 

I just need to take live source streams as an MP3 and broadcast them to the world while recording them on the server the instant the source goes live (a couple times a week). If it's reliant on a script that runs, even every minute, there's still a good chance that the first bit of the broadcast will be cut off. I'm about done with all the research I've been doing the last few days. :retard:

 

I really don't care what broadcasting server application is used. Freeware, FOSS, or paid. I'd love it if it was relatively user friendly as I'm the most technical person on this project and I'm only here temporarily so other less techie people will need to muck with the addition of new stations that come on board. It's a pretty rural outfit. I'm trying to steer clear of shell scripts and XML files, but if that's how it has to be, that's how it has to be.

 

 

Misc. Notes:

For simplicity's sake, I'd like to keep the server as Windows. Its what this shop knows. I think I know the most about Linux, but I'm trying to avoid a bus scenario. E.g. If I get hit by a bus, this place is up a creek without a paddle, at least until someone can devour Linux for Dummies. :laugh:

 

Thanks so much for taking the time to read this! =)

Edited by Nonapeptide
Link to comment
Share on other sites

"I'm working with a internet radio business that sets up encoders at rural radio stations in the US. The rural stations take simple line-level audio from their boards and plug them into a workstation nearby. The workstation is using Windows Media Encoder 9 (I know, I know...) to send a WMA 8 stream back to a Windows Media Services 9.6 server."

 

Hmm, you sound like a stream host. most stream host dont use the word rural, so this makes you a little different. But the way you presented the idea of what you do, Im not really catching what makes you different just yet.

 

"The rural stations only record very specific shows, so it's not like a 24/7 stream. The streams last for about 2 to 4 hours. Each radio station might have a few broadcasts each week."

Ok, they sound like hobby broadcasters like myself... Go on...

 

 

"On the WMS Server there are about 250+ total publishing points, one for each radio station. Each station has its own publishing point. Listeners go to oursitename.com/call-letters to get to the stream for their radio station. Here comes the most important part: Each stream has the "WMS Archive Data Writer" plugin enabled and saves the live streams to a file."

 

Ahh, Ok, Im seeing a clearer picture right now. Its like they are recording their show for podcast, and instead of submitting it to a place like I-tunes that would require licenses, they are using a site to submit content. I have a friend that would love to be a client, message me.

 

 

 

Now this is where you confuse the crap out of me. Are you trying to re stream their show? So instead of a podcasted archive it just loops as a live stream without them streaming? If so WHY, oh WHY would you want to use a WME? for 250 Stations, WMS's are allready so resource intensive. You would be better off going server edition of Ubuntu or REDHAT WHM.

 

If you can't tell already, Im not a big fan of WME. Any other format is so compatible now, and flash can do so many things that WME cant do.

 

"we want to stream to as many devices as possible, namely mobile devices."

 

Tune-in does them all. Look it up. Many other apps as well that are multi platform, but this one is by far the best. Do it as an app, or your clients will never get a good listener count, and blame you, as all the listeners phones go into sleep mode and shut off audio. Permissions must be made.

 

 

break it down for me a little clearer, and I might be able to help you. But for WME? why oh why....

KNSJ.org / 89.1 FM San Diego
Link to comment
Share on other sites

Hmm, you sound like a stream host. most stream host dont use the word rural, so this makes you a little different. But the way you presented the idea of what you do, Im not really catching what makes you different just yet.

 

Really, there's not much different. This place doesn't give out control panel access, or really anything. The radio stations, like I said, are rural. Well, some are more rural than others. As word has gotten around over the years this place has picked up some decent sized stations, but still within the small radio station market.

 

The attraction is personal trust. Mom-n-pop stations trust this place. They pay nothing to us. We send them instructions, they set up WME, point the encoded stream to us, and we host the stream in our station list. The radio stations don't do anything else. They don't want to. They don't want control panels and statistics and graphs or any of that.

 

The specific twist to this business is that we only broadcast one type of show. So when the station starts to broadcast the show, they turn on the encoder that's plugged into their board (usually just an ancient Windows XP box running WME with line-out plugged into a soundcard). The encoder sends the stream to us for two, three or four hours until the show is over. Then they turn the encoder off, and the stream is gone. The radio station continues to broadcast it's 24/7 rotation of shows and music, but we don't care about that. We only care about one type of show so the stations only send us their broadcast in that genre.

 

As far as WME itself is concerned, we'll have no problem telling the stations to uninstall WME and install a new encoder. We'll make a new set of instructions and show them how to do it. We're completely ready to do that. In fact, for every station that wants an MP3 stream rather than a WMA stream, that's exactly what they'll have to do. No exceptions. I'm looking at VLC to be the encoder to send the MP3 stream to our stream host server.

 

That's where the sticky wicket is. What to use as the server? IceCast? ShoutCast? Something else? If it wasn't for those darned archives that we need to take. More information on that below...

 

(If you can't tell, I'm new to this organization. In fact, I'm a hired contractor. I'm an independent SysAdmin that focuses on Windows / Linux servers with some networking and storage thrown in. I was hired for my Windows experience to work on access controls, server tweaking, and etc. but the specifics of the broadcasting world I am left to discover on my own.)

 

Ok, they sound like hobby broadcasters like myself... Go on...

 

They're legit radio stations that broadcast 24/7 on AM or FM. They have morning traffic reports, talk radio, sports, music, the whole shebang. We just only care about one genre of show.

 

On the WMS Server there are about 250+ total publishing points, one for each radio station. Each station has its own publishing point. Listeners go to oursitename.com/call-letters to get to the stream for their radio station. Here comes the most important part: Each stream has the "WMS Archive Data Writer" plugin enabled and saves the live streams to a file.

 

Ahh, Ok, Im seeing a clearer picture right now. Its like they are recording their show for podcast, and instead of submitting it to a place like I-tunes that would require licenses, they are using a site to submit content. I have a friend that would love to be a client, message me.

 

I didn't really explain myself well. The stations aren't recording anything to disk. We record/archive on our end at the stream host servers. That's our gig. People who want to listen to the program that they like, but missed it due to time differences (we're big with service members who are abroad), can pay for access to our archives and listen to whatever they missed.

 

Interestingly, the stations don't really care. I mean, they'll talk to their listeners about it maybe if the station thinks its a service their listeners would like, but few do. The stations are just happy to have that genre of show streamed to the internet. We need to make some more money, so record the show for people to play back if they're in East Krapistan and were sleeping or whatever while the show was on.

 

Now this is where you confuse the crap out of me. Are you trying to re stream their show? So instead of a podcasted archive it just loops as a live stream without them streaming? If so WHY, oh WHY would you want to use a WME? for 250 Stations, WMS's are allready so resource intensive. You would be better off going server edition of Ubuntu or REDHAT WHM.

It's just taking their live stream and hosting it so they don't hose their bandwidth. We got a great deal on bandwidth. So in essence, yes, we're just a stream host. We take the selective stream they send to us (just when that specific genre of show is on), archive it on the fly and simultaneously stream it to whomever wants to listen to the live broadcast. If people missed it the broadcast, they have to subscribe to the archive serve that we operate to listen to the old programs.

As far as WME/WMS...

If you can't tell already, Im not a big fan of WME. Any other format is so compatible now, and flash can do so many things that WME cant do.

Oh, you don't like WME? Well I'm the chairman of the DESTROY WME WITH EXTREME PREJUDICE board. I am thoroughly disgusted with the entire stack of Microsoft tools for streaming either audio or video. I hate WMA format. I hate WME. I hate .asf files. I hate MMS. I hate Microsoft's attempt at HLS. I hate Smooth Streaming. It's all just a debacle of legendary proportions.

 

The only thing in all of this that seems halfway decent is WMS itself if you can deal with WMA streams. We've had tens of thousands of simultaneous listeners on heavy nights and the servers yawn. Bandwidth is big, but virtually no CPU usage. After all, we're not transcoding or muxing or anything. Just slinging the station's streams and archiving the encoded stream that the station sends to us.

 

we want to stream to as many devices as possible, namely mobile devices.

 

Tune-in does them all. Look it up. Many other apps as well that are multi platform, but this one is by far the best. Do it as an app, or your clients will never get a good listener count, and blame you, as all the listeners phones go into sleep mode and shut off audio. Permissions must be made.

I'm looking at tune-in now at your mention of it. I'm not seeing where it fits in for us, but that's probably because I didn't explain things well above in the first post.

 

In essence, we want to be the station's stream host since they like us, trust us, and we understand the small market radio business. We just want to send their streams out as an MP3 stream, not WMA like what's being done now. The trouble is, part of how we get paid is archiving the streams when they're live. Just one or two shows that the radio station broadcasts per week.

 

So we need to find a streaming server system that can simultaneously host a stream and archive it to disk. Who would have thought that Windows Media Services would actually have a superior product in some way? It seems like this simple feature is really hard to come by in other products.

 

break it down for me a little clearer, and I might be able to help you. But for WME? why oh why....
We're not married to WME. We'll tell the stations that want to move to MP3 streams that they need to uninstall WME and use another encoder.

 

 

Wow, I type a lot. Thanks for reading this far! :thumbup:

Edited by Nonapeptide
Link to comment
Share on other sites

What about Wowza ?
Thanks for the tip! I forgot about Wowza. I found them in my search for media servers. They're expensive, but it would be worth it if it just does what we want in a rock solid, repeatable fashion.

 

However, I don't yet see any mention of an archiving feature for when our remote encoders send us their stream. :sad:

Link to comment
Share on other sites

Now this makes complete sense.

 

The easest way for them to do this?

 

Their are two standalone programs that will record their stream from source when the use the encoder to stream, and they can ship it directly to you or drop box it for your convenience.

 

EdcastStandalone _ Edcast reborn project.

 

It has the ability to record from the time they start streaming, and can save to archive.

 

The One thing you have to let your customers know is that they need to create a file in their computers directory if they use Win7. It was designed for XP.

 

The second? And my personal recommendation, is BUTT (broadcast using this tool) from sourceforge.

 

VERY SIMPLE to set up, VERY lightweight on the processing, and It was made within the last 2 years, so it will work with current setups well.

 

Set up a dropbox, and just upload them from there. Or you can streamrip using a ripper with either.

 

The nice thing is Edcast Reborn project also has an Edcast AISO build for soundcards like SB audigy/X-FI to really take full advantage of their abilitys.

KNSJ.org / 89.1 FM San Diego
Link to comment
Share on other sites

OH! The server. Icecast2 KH build.

 

The nicething about Icecast is You can run a server on a /mount point file extention. That means all your streams are pointed at one port, and one display page for all of them, and designate them by mount. Then you give them the mountpoint, and they can stream any codec, at any bitrate, at any Khz, and the server will support it. They also offer a windows build through the KH, and setting up the config file is fairly easy if you know anything about networking. GOT TO BE KH ICECAST (Karl Heyes trunk)

 

But you can set up a pretty fluid server for ripping the stream in this mannor. Wont have to open a TON of ports either, since the server runs off one port and designates the streaming source by mountpoints.

KNSJ.org / 89.1 FM San Diego
Link to comment
Share on other sites

Now this makes complete sense.
Whew. I was worried I wasn't making sense. :retard:

 

The easest way for them to do this?

 

Their are two standalone programs that will record their stream from source when the use the encoder to stream, and they can ship it directly to you or drop box it for your convenience.

 

You know, I had considered recording at the source. I mean, it makes the most technical sense. I like your recommendation of Edcast reborn. I've been considering using that as our source client anyway, regardless of logging features. The whole "create the file first" thing seems remarkably flaky though, and with some of these stations I'm not sure how we'd even explain it to the very non technical people there. hmmmmm.

 

What troubles me is that many of these stations are already either extremely non-technical, allegedly too busy to do more than click one button in one application, and we also need the archived broadcasts within 24 hours or so to meet our own listener demand. So physical shipping is out. Dropbox is great, but of course now we've got to tell each of these stations to install another program, make another account, point the archive to the dropbox...

 

I realize that this is more of a personnel issue, rather than a technical issue. I'm trying to solve a people problem with computing skills. I just don't know how else to get around this though.

 

The frustrating part is that the archiving process is working great right now with WMS. I never thought that such a seemingly simple server-side feature would in actuality be so hard to come by in other products. :sad:

 

 

OH! The server. Icecast2 KH build.

 

The nicething about Icecast is You can run a server on a /mount point file extention. That means all your streams are pointed at one port, and one display page for all of them, and designate them by mount. Then you give them the mountpoint, and they can stream any codec, at any bitrate, at any Khz, and the server will support it. They also offer a windows build through the KH, and setting up the config file is fairly easy if you know anything about networking. GOT TO BE KH ICECAST (Karl Heyes trunk)

 

Yeah, that was one of the biggest reasons I chose Icecast in my mind, rather than Shoutcast. It works a lot like WMS in that you can run virtually unlimited stations off of the same port with the use of mount points.

 

But you can set up a pretty fluid server for ripping the stream in this mannor. Wont have to open a TON of ports either, since the server runs off one port and designates the streaming source by mountpoints.

 

Okay, tell me how crazy this idea of mine is. I was considering leaving the current WMS servers untouched. They're getting WMA streams from source and then the serves are hosting them. What if we got a third server, running icecast or something, that took the mms:// WMA streams from the stream host servers and transcoded WMA to MP3? It's 32k streams that we're dealing with, so nothing huge. I think that would kill the CPUs though, of course. Since we only broadcast a certain genre, we might have only 10 streams live, but on some nights and seasons, we can have 70 or more simultaneous streams from radio stations.

 

Ergh. That doesn't sound so good now that I've said it out loud.

 

Where I'm at now is looking at IceCast and it's on-connect directive for a stream. I think I can call a wget script for each stream. It's going to suck for a month while stations get up on it and there's a lot of manual frobbing around. *sigh*

Link to comment
Share on other sites

STOP THE PRESSES!!

 

I had an epiphany. First, the source client software will need to be changed no matter what happens. That's been a given almost the entire project.

 

However, what if the source client can take the audio source (in this case line-level audio from a sound board) and then encode it into two separate streams. One a WMA stream to the WMS servers, the second an MP3 stream to the icecast server. The WMA/WMS stream is purely for the archive feature of WMS. The MP3 stream is purely for our listener's benefit to get mobile access to the audio.

 

"Just try it out yourself, lamer!!" I can hear you yelling. I'll get there hopefully today, but thought I'd toss the idea out there anyway. :cool:

Link to comment
Share on other sites

Currently looking into Streaming Media Hosting.com to see if they can just handle it all for us. One unified system, one point of contact... might be worth it.

 

I called Wowza's sales department and left a message this morning. Almost 5PM and no callback.

 

While I wait for sales people and techs to call me back I'm going to play with IceCast. Still not liking the idea of a hacky wget script... :eyebrow:

Link to comment
Share on other sites

You can do that. If it was me? I would pay for a team viewer license. Then all they have to do is install Team Viewer 7, and their done. You take over all the footwork for them, and have access to their computers to do updates if neccessary.

 

This sounds hard, but drop me a line to skype. Trust me on this.

KNSJ.org / 89.1 FM San Diego
Link to comment
Share on other sites

Currently looking into Streaming Media Hosting.com to see if they can just handle it all for us. One unified system, one point of contact... might be worth it.

 

I called Wowza's sales department and left a message this morning. Almost 5PM and no callback.

 

While I wait for sales people and techs to call me back I'm going to play with IceCast. Still not liking the idea of a hacky wget script... :eyebrow:

 

Keep in mind WOWZA is developed in another country (like the netherlands or Holland, the support staff has a Saxon accent. I believe the number for the US redirects to that country via SIP VOIP services.

KNSJ.org / 89.1 FM San Diego
Link to comment
Share on other sites

Don t do it. Call me on skype.

 

Brutish_Sailor on Skype. Add me to your contacts.

 

Done, and done. You're a gem.

 

You can do that. If it was me? I would pay for a team viewer license. Then all they have to do is install Team Viewer 7, and their done. You take over all the footwork for them, and have access to their computers to do updates if neccessary.

 

This sounds hard, but drop me a line to skype. Trust me on this.

 

Yeah, a while back I was looking at remote access tools for my consultancy. TeamViewer and Bomgar were the two rockstars with LogMeIn Rescue a close second. I've still accruing assets to get a Bomgar box. Anyway, yes, I told my client that he could use TeamViewer to have instant access to all their PCs. I think he was uninterested in being that easy to contact by the stations lest they abuse that. I dunno. I think it's a great idea.

 

You and I think a lot alike. Do you contract on the side? =)

 

Keep in mind WOWZA is developed in another country (like the netherlands or Holland, the support staff has a Saxon accent. I believe the number for the US redirects to that country via SIP VOIP services.

 

Wait...wut. I saw on their site that it was Evergreen CO. That was actually what attracted me to them (or rather, one of the things) because I know people not far from there. Always nice when you buy something that you can visit the offices of. =)

 

But yeah, you know more about that whole thing than me. Then again, I have people in Amsterdam and Utrecht, so... =)

 

 

I just put in a call to EvoStream to call me back tomorrow. Man, tomorrow is going to be hectic. Engineer call with SMH, Wowza should respond back, now Evostream. I've looked into red5 and considered calling Infrared5 for some advice.

 

The thing that annoys me is that most of these products that are so awesome focus mostly on video. I know, I know... video is where it's at. But I want audio dagnabbit!! It's like audio is a distant cousin of video streaming. There are some of us that still like non-visual media, people! :cool:

 

*sigh* Once I get all this figured out, I'mma hafta come up with a plan to develop a product that's what I needed in the first place. I'm taking venture funding offers now... :lol:

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...