A Laggy Problem in Foxworth Part 2: “It’s Your Fault”

Recently, Foxyfurman Kunami had some trouble with lag on his sim, lag that turned out to be from a sim that was basically a Shoutcast server farm. Although Linden Lab eventually took care of the problem, it was after a friend of his found it and alerted them. In the meantime, it appears some in LL kept pointing the finger back at Foxyfurman, as this note sent by him suggests.

* * * * * * * * * *

Bix thought you would find this entertaining, the "it's your fault" response by LL. I do want to add as a side note when I called LL I got Gareth Ontyne on the line, by moving my ticket someone took care of this the next day, so the problem was fixed and everyone knows what happened, it is obvious that the shoutcast server was basically locking up all the performance, but the "canned response" in words is just stupid. I commend the actions but the BS that "blame the sim owner" is just so stupid it is funny.

* * * * * * * * * *

Foxyfurman:

Re: Huge Lagg issues on my sim

Spontaneously about a week ago we have had HUGE lagg spikes, we have tried shutting down scripts section by section, with zero help, your service desk told me to re compile all my top scripts into LSL, this is NOT the problem, and it will break many items. We have shut down section by section we still have huge problems, basically my sim is un-useable at this time. Again I have tried shutting scripts off section by section and the only way it even works OK is when ALL scripts are off, even a few scripts will over tax the sim. I have NEVER in a year had this issue and have added nothing new, yes I have restarted, many times including a LL initiated re start. I was informed by my manager that this could have started mid Jan.(ish)but just not as bad, not sure who is new sharing our server but this is a huge mess, thanks for taking care of this ASAP.

* * * * * * * * * *

Gareth
Linden Lab Support:

Hi Foxyfurman,

Thanks for contacting us in regards to this issue.

I am going to escalate this ticket to the appropriate department and they will contact you shortly via email regarding your request. In the meantime, if you need to contact us again about this issue, please add a comment to this ticket rather than creating a new ticket.

* * * * * * * * * *

Spike Linden
Linden Lab Support

Hello Foxyfurman,

Thank you for your ticket regarding your region and the performance issues.
I'm sorry to say that if there are lag spikes in the way you have described and they continue after restarts, then the source of the problem is within your region. As you have scripted spikes, then it will be scripted items within your region causing this.

Every time you restart a region, it will change server, so other regions can not keep causing issues if you move from host to host.

How were you disabling scripts section by section? This can't actually be done completely.
Unchecking the Run Scripts option in About Land will only disable scripts for non-parcel owner. You can't actually disable scripts for items that are owned by the parcel owner. So if a parcel is deeded to a group, all deeded items will still be running. If a parcel is owned by an avatar, then all items belonging to that avatar will still be running, with both 'All Residents' and 'Group' unchecked for Run Scripts.

This can help you narrow down the problem though, as you can eliminate most parcel/objects through this method.

I've had a look at the region today and script time is high (18-20ms). The chances are high that as avatars go to the region you will be running low on available resources and start to have issues. In this case your 'only' option is to look at your content. You will have to either re-script more efficiently or remove items completely from the region. There is no other choice for this type of issue.

Please make sure to look at our wiki page for 'Improving Region Performance'. This page can help you greatly in managing your region better. Key thing to remember is that you can't just fill a region with anything you want and expect it to work. You 'must' manage your regions content effectively.
TICKET CLOSED

* * * * * * * * * *

Ticket RE OPENED

Foxyfurman:

This was fixed prior to your investigation :)

Someone from LL moved the shout cast server sim that was running on the same CPU and like magic everything is totally fine. We are crazy careful about what we put down and paranoid about Time Dilation and our total script time. With a shout cast sim as a "virtual neighbor" our time dilation was 70 now it is 99. In addition for fun we ported to these shout cast sims and all of them were horrific with lag/with time dilation in the 80-50 range and all of their virtual neighbors were a train wrecks as well regardless of ANY avatars.

We had shut down all scripts in the estate manager panel and still had lag this is how we know to look OUTSIDE our sim. It is not always the sim owners fault as you say.

Sorry I re open this ticket but this caused my sim to basically stop in its tracks, it will never happen to me again because I will know what to look for, but I know others will not be so creative in looking outside their sim for the problem.

* * * * * * * * * *

It seems persistence is in order when dealing with the Lindens. Fortunately, Foxyfurman has plenty of it.

Bixyl Shuftan

A Laggy Problem at Foxworth

Recently, the lag at my home sim at Foxworth became particuarly bad, just after January 20. Sim owner Foxyfurman Kumani was among the first to notice, saying he found he was barely moving, and the only others in the sim at the time were five in the Mayan Casino, “It was not long after a Linden Lab restart of the sim.”

Explaining the problem in a group notice, “The first two calls I had to Linden Lab were a total waste of time. One suggestion (of theirs) was to restart the sim. (laughter) Okay, this did not work. The other was to look at our top scripts. We already did this, and even returned new scripts that had been put out during the period of time when the problem started. We tried turning off scripts section by section (in the sim) with no positive results.”

Both his sim manager Aiko Swashbuckler and the sim technician Leonarris Orsini helped Foxyfurman in looking for the problem. Not having found anything, Aiko asked everyone living in the sim to stop tinkering with scripts. But as it turned out, none of the tenants was behind the problem either, “no one in Foxworth did anything to cause this.”

So what caused the intermittent lag? Leonarris did some investigating and finally found the source on Friday Jan 29. The problem was the way sims are stored within Linden Lab’s servers. A sim doesn’t have to be physically next to others on the map and Grid to cause problems. If two sims next to another in storage, what happens in one can affect the other. Aiko explained Foxworth was one of “ several sims on one server” with “three Class Five ones. One of our ‘neighbors’ on the same (server) was running a Shoutcast server farm business.” Foxy went on, “a new place just came online a week or so ago. ... It is so laggy that you can’t even walk at that place, and it is sucking resources from the machine. ... Linden Labs knows of the problem and will fix the recourse allocation or move us to another (server).” He stated Foxworth’s place on the map would not be changed, at most it would “just run on a different machine.”

The following day, Foxy made another group notice, “Linden Lab has moved the offending sim to another server. if you have noticed, you can move now and walk and fly, all good things. Thanks to everyone who helped, and those who suffered the lag along with us.” And everyone was in complete agreement, “it was a nightmare,” “yeah that Shoutcast thing was crap, I'm glad they took care of it so fast,” “Indeed, the lag was horrid. O.o “

And so, peace and harmony was restored to our little corner of the Metaverse.

Bixyl Shuftan

Sion chicken a menace to sims as they are..........

Today I had an IM from Intlibber Brautigan, he said he had a story for me and asked if I knew about 'Sion chickens'
Actually thanks to Gemma's story a few weeks ago I did but not very much and I said so.
IntLibber Brautigan: they lag the crap out of sims, physics and script lag. They've been spreading like an infection across the estate - I spent last night dealing with lag complaints all over the estate
IntLibber Brautigan: 10 chickens make 1 ms of script lag but each chicken makes 150 potential collisions!
He was so infuriated he considered an AR against the creator citing griefing issues but instead decided to speak to the designer and point out the problems.
Sion Zaius worked with Intlibber to fix the issues and had this to say:
Sion Zaius: yes, there have been problems with sion chicken causing physical lag. I'm working on the update which is called "version 11 lagless"
Originally the chickens were updated from the feed but this was changed about a week ago so I asked Sion what people needed to do as I understand you can't just pick them up and re-rez them.
Sion Zaius: since you cannot pick up chickens or eggs into your inventory without breaking them, you have to use chicken transport boxes or proteggtors to do so, if you use those objects, your chicken/egg will be updated automatically.
Sion Zaius: then, people would have to box up living chickens once, and free them again lag-improvement should then occur instantly - its awesome, this chicken has as collision score of 3.4 ... it had 140 before
IntLibber Brautigan: We will be requiring all of our residents who have chickens to update them if they want to keep them.
The lag improvement is just tremendous, its going to improve sim life for everybody.

To find out more about the chickens and why they are so popular I spoke to Phiona Ember who loves the chickens but also sees the enterprise opportunity too:
Phiona Ember: The new update will make it easier to have more chickens in one place and make it easier to update the chickens and care for them.
i love them :) they are fun to watch grow.. and breed, believe it or not they are very therapeutic.
dana Vanmoer: I guess they could get addictive
Phiona Ember: omg VERY
dana Vanmoer: so you keep a male and female together?
Phiona Ember: yes so they can breed, once you hatch an egg it takes about 2 weeks till they are of breeding age.
then you can pair them up.. and they make more eggs :)
dana Vanmoer: what do you do once you get too many though?
Phiona Ember: you can sell the eggs or keep them.. same with the chickens :) there's always more people that may want the colors you have or may want to start their own little farm :)
dana Vanmoer: So you can start trading them once you get started
Phiona Ember: many people, myself included, build a small business from it
Phiona Ember: some people are all about the business some people just enjoy the chickens I guess I'm lucky for me its a bit of both ..

The update should be ready by the end of today - Sion recommends all chicken owners update as soon as possible to avoid lag on your sims.
Sion Zaius: Special thanks to Intlibber, who made it possible for me to discover a way to reduce collision lag

Dear Linden Labs ...

WELL, IT'S FINALLY HAPPENED - I've joined the ranks of the people that you've pissed off!

Click here to read what has got me so damned mad.