What is "Bose Monitoring Service" User Agent

[joint]

Hi

 

I see on my SHOUTcast Server a user agent called "Bose Monitoring Service"

It is constantly connected to the SHOUTcast server.

 

what does it do?

 

Thanks

Shai

[KatsMetalLitterBox]

Just got them on my station today. I think it's a bose product. Vtuner is available on Bose products so it could be that not 100% sure. Connected for almost 2 hrs and still connected.

[joint]

The agent stay connected all the time

[saint]

Perhaps someone is listening to your station all the time ?

[GKIye]

After a search into several forums, also those in the German language I learned that it is a ""Monitoring Service" of "Amazon Web Services"

This should be the link to their website :

https://aws.amazon.com/

 

However, for reason of the brand name "Bose", it doesn't point to a German IP but to an USA located IP

Very common should be this one :

United States Ashburn 54.205.91.122 Bose Monitoring Service

[KatsMetalLitterBox]

It's bose the company doing it I found out. Going after them right now for costs of bandwidth and time wasted.

[robert4u]

Its possible they are monitoring your shoutcast serveras the name is showing.

[HowickNZ]

Here is their response after I emailed them:

 

We have a service that monitors web radio streams that our customers listen to on Bose devices. We log the metadata that is provided in the audio stream for on-screen display and stream profiling purposes, we are not ripping, relaying, or storing the audio content in any way. The purpose of this service is to help our users find the web radio streams that they will enjoy the most. Our systems only monitor publicly accessible stations using a catalog provided by a company that indexes web radio streams.

 

We have the ability to permanently remove streams from our systems on request. Let us know the name(s) of any stations you would like us to block. Alternatively, you can disallow the "Bose Monitoring Service (BoseMonitoringService@bose.com)" player from accessing the stream or ignore it in your reports reporting services. User controlled devices that are listening to the stream do not identify as "Bose Monitoring Service" so there is no impact on human listeners if you block the monitoring client.

 

[Roger Hågensen]

I got the same answer in February. And one of the guys at Bose said that they would change the way they get their metadata but could no give a ETA. Here is the exchange so far:

 

Hi!

 

I'm the Technical Consultant for gridstream.org a internet radio station. We have notices a *"Bose Monitoring Service" connected 24/7 to our radio server (via Amazon AWS)

What is this service? And why is it connected to the stream 24/? Is this related to your products? If it is a cloud service Bose runs that re-broadcasts the stream to the devices of all your customers then you may get in legal trouble with the Performance Royalty Organizations and the RIAA.

If this is not the case then your monitoring services need to be fixed as it must be connecting to the wrong url causing the Shoutcast v2 server to respond with the stream instead of status information. We have contacted Amazon AWS abuse centere and are awaiting answers from then, we'd rather not have Bose Monitoring Service banned if it turns out this is a legit single listener.

Roger Hågensen

Roger,

 

We have a service that monitors web radio streams that our customers listen to on Bose devices. We log the metadata that is provided in the audio stream for on-screen display and stream profiling purposes, we are not ripping, relaying, or storing the audio content in any way. The purpose of this service is to help our users find the web radio streams that they will enjoy the most. Our systems only monitor publicly accessible stations using a catalog provided by a company that indexes web radio streams.

We have the ability to permanently remove streams from our systems on request. Let me know the name(s) of any stations you would like us to block. Alternatively, you can disallow the “Bose Monitoring Service” player from accessing the stream or ignore it in your reports reporting services.

-Dave

 

 

 

 

On 2017-02-24 15:45, Datta, David wrote:

>

> Roger,

>

>

>

> We have a service that monitors web radio streams that our customers listen to on Bose devices. We log the metadata that is provided in the audio stream for on-screen display and stream profiling purposes, we are not ripping, relaying, or storing the audio content in any way.

 

 

If you are not using the audio data but just the song artist and title then there is a proper API for this with Shoutcast v2 servers that you can periodically poll and reduce bandwidth costs.

 

 

For example for current artist/title info as JSON data http://5.152.208.98:8014/stats?sid=1&json=true

or if you prefer XML http://5.152.208.98:8014/stats?sid=1

 

 

And song history (last 10 songs) as JSON data http://5.152.208.98:8014/played?sid=1&type=json

or if you prefer XML http://5.152.208.98:8014/played?sid=1&type=xml

 

 

These two info urls are reliable and I now poll these myself for (the upcoming revision of our) own internal tool that we cache and then provide to assist our Windows desktop player and to our Webplayer which is based on HTML5 audio and don't support any shoutcast embed metadata, that cached info can also be provided for other purposes like displaying on a Forum or Website.

Do note that in both examples above the info is only for Stream ID 1, we only have a single stream on that server address. Other stations however may share a server and in that case you need to use the correct sid number.

 

 

In theory you could poll once each 300 seconds (10*30sec), assuming that no song is shorter than 30 seconds each and still have a fairly accurate history, the SHoutcast v2 played history ecen provide a timestamp allowing correct duration calculation.

The data is less than 1KB in size which means it fits inside 1 single TCP/ethernet packet, making this retrieval lightning fast (literally).

 

 

Details like year or album is not normally embedded in the stream, only exception is if the DJ broadcast using Shoutcast v2 to a Shoutcast v2 server.

But DJs may choose to use either Shoutcast v1 or Shoutcast v2 so there are no guarantees for that.

Year and Album titles (and any other extra metadata) may not always be correct or empty/missing entirely as we do not require our DJs keep those fields accurate.

Only the "Artist - TItle" pair is required to be correct (mostly for logging and royalty reasons).

 

 

For Shoutcast v1 servers there is no played history to retrieve (unless you wish to scrape the HTML page and parse the info from that).

Getting current song is easier though, for example http://5.152.208.98:8014/7.html

As you can see that url still works on Shoutcast v2 (for Backwards compatibility with old tools) just like it does on Shoutcast v1.

 

 

More info here http://wiki.shoutcast.com/wiki/SHOUTcast_DNAS_Server_2_XML_Reponses

 

 

Do note that artist/title info may not be accurate, a DJ or a server may choose to stagger the embed stream metadata so they do not sync perfectly with the audio stream to help deter stream ripping (if a streamripoer missed the start or the end of a song it's useless for most people trying to rip from the stream).

 

 

Also note that Icecast is a little different and does not have a played history in XML or JSON format (out of the box), there exists modified Icecast software that might have such extra info available.

 

> The purpose of this service is to help our users find the web radio streams that they will enjoy the most. Our systems only monitor publicly accessible stations using a catalog provided by a company that indexes web radio streams.

We do not have a issue with listeners using a Bose device as long as that Bose device show up as a 1 regular listener per customer/user. What Useragent ID does the listener device use when a listener is connected?

 

 

I am not happy with the Bose Monitoring Service eating up 1 listener slot as that is not needed to get the information it needs (certainly not with Shoutcast v2), I'd consider that really bad engineering/design.

 

 

Please note that I have contacted (Prior to your reply) Amazon AWS and inquired if this could be a rogue script or excessive web crawling (which this would classify as) and could be a potential violation of their terms of service, we have not received any answer from their abuse department yet.

 

 

Also note that Bose Monitoring Service since it grabs the stream 24/7 counts as a listener tuned in 24/7, which means we have to pay royalties for all the songs that that Bose Monitoring Service "hears".

If we are going to pay royalties we'd rather pay that for an actual listener. The server has a max amount of listeners it can server, since Bose Monitoring Service is tuned in 24/7 that number is max-1, this also means that you unintentionally gobble up a stream slot that another BOSE customer could have connected to example. Now we pay for total slots, other stations may pay for bandwidth in which case you are incurring a undesirable overhead that should have been spent on actual listener.

 

 

We'll tolerate Bose Monitoring Service for now, but not for too long. We'll wait until the end of March for you to improve your Bose Monitoring Service to fetch the data the proper way rather. If by then we still see Bose Monitoring Service in the logs fetching the entire stream we'll ban your service. Something not we, you nor your customers (and our listeners) want.

 

 

Considering how many radio stations there are out there you can imagine the headache it would be if all those stations wanted to re-claim (from Bose) the royalties unjustly paid to the PROs due to the Bose Monitoring Service "listener". That's 175320 songs per year (assuming 3 minute average song length), now I don't recall the exact royalty rates per song (and it varies from station to station) but you can imagine that amount and then multiply it by the number of stations the monitoring service is using a listener slot on.

 

> We have the ability to permanently remove streams from our systems on request. Let me know the name(s) of any stations you would like us to block. Alternatively, you can disallow the “Bose Monitoring Service” player from accessing the stream or ignore it in your reports reporting services.

>

> -Dave

 

 

I hope it won't come to that, we'll see by the end of March what we'll have to do.

 

 

Regards,

Roger.

 

 

 

On 2017-02-27 12:41, Datta, David wrote:

Roger,

 

Bose devices don’t use the user agent “Bose Monitoring Service” when connecting for listeners. I don’t know what the user agent is but it is absolutely not this one.

 

I cannot commit to a timeline when we would be able to change how we are gathering this data. Please let me know which streams we should remove from the service and I will contact you after a change is in place.

 

-Dave

I spoke with our station owner and he says that currently Bose Monitoring Service is not a financial burden currently and is OK with it for now.

 

I understand you cannot commit to a timeline, but 1-3 months (development and testing) should be realistic I assume?

 

BTW! For Icecast servers information on the stream can be found at http://127.0.0.1:8000/status-json.xsl?mount=/live

Where "live" is the mount point.

More info about that here: http://icecast.org/docs/icecast-2.4.1/server-stats.html

 

Unfortunately Icecast does not have a song history provided so in this case you would need to poll status-json.xsl more frequently, 30 second intervals should be enough to make sure no songs are missed (though some jingles or station IDs could be shorter than 30 seconds but listeners usually do not wish to know the name of a jingle). And assuming Bose caches the info in your own system before handing it to the listener devices the stress on the server is minimal.

 

Please note that I will be contacting the Icecast devs to suggest adding a history stat which can be polled instead, this would allow a less frequent polling rate, fully accurate data and timestamps (allowing showing of song start time and duration). Icecast development is somewhat glacial (pun intended) so such a feature probably will not be available until Icecast v2.5

 

When it comes to old Shoutcast v1 there is no way to get info a proper way, but since Shoutcast v1.9.8 is frozen in time and won't ever change one can safely scrape it's webpage for info like station name and songs.

On old Shoutcast v1 servers you need to scrape http://127.0.0.1:8000/index.html and http://127.0.0.1:8000/played.html

 

Please note that both Shoutcast v1 and Shoutcast v2 has a issue/flaw regarding the song start times, the times are supposed to be Unixtime time (UTC timezone) but the HTTP headers do not include server date so there is no way of knowing if the server clock is accurate which reduces the usefulness (as there is no clock reference) of the start time of the songs, duration of songs can still be calculated with the info though. I'm working on a script for my own use that does just that.

 

And today/yesterday I sent this (waiting for an answer now):

It has been months since we notified you that you are using up a listener slot and consuming that bandwidth 24/7.

 

We suggested then that you'd use the appropriate urls and APIs to query the metadata from Shoutcast v1, Shoutcast v2 and Icecast servers.

 

I understand you can not commit to a timeline, but it has no been half a year. Nobody program code that slowly.

A rough data polling prototype should be possible to do within a work day.

 

Our current server(s) can be found listed in player.gridstream.org/stream.m3u

 

If our stream urls or ips should change then that will be reflected there.

 

It is a wonder you guys haven't had a class action lawsuit yet considering you have incurred thousands of additional royalty payments that stations do have grounds to demand reimbursement for.

 

Roger Hågensen

[djaubs89]

bose monitoring service won't track online radio and have us arrested will they?

[djgary72]

bose monitoring service won't track online radio and have us arrested will they?

 

Only if you're running your station as Pirate... Then you might run the risk of being arrested and having you computers taken.

 

I don't think there's anything sinister in this user agent myself, But then again User agents can sometime be faked.

[kapital3]

"Amazon Web Services" I have visited on my live camera website , and now with shoutcast server v2 "bose monitoring service"

I will try bloc his IP and see what happen.....

[James]

"Amazon Web Services" I have visited on my live camera website , and now with shoutcast server v2 "bose monitoring service"

I will try bloc his IP and see what happen.....

 

A quick Google will reveal what AWS is.

I would advise against blocking these IP's as they collect data useful for listing/playing your station on various services/devices.

Bose Monitoring Service is used to monitor your streams. If you block or limit this user agent, your stream will not appear in the Base Soundtouch service. AWS is used by over 35% of the web (services such as Netflix and other large companies, Broadcasting World even uses it!).

[James]

The BW monitoring service de-lists any stations that are not active. If you block the BW monitoring service, your station will no longer be listed on our website.