A Laggy Problem in Foxworth Part 2: “It’s Your Fault”

Recently, Foxyfurman Kunami had some trouble with lag on his sim, lag that turned out to be from a sim that was basically a Shoutcast server farm. Although Linden Lab eventually took care of the problem, it was after a friend of his found it and alerted them. In the meantime, it appears some in LL kept pointing the finger back at Foxyfurman, as this note sent by him suggests.

* * * * * * * * * *

Bix thought you would find this entertaining, the "it's your fault" response by LL. I do want to add as a side note when I called LL I got Gareth Ontyne on the line, by moving my ticket someone took care of this the next day, so the problem was fixed and everyone knows what happened, it is obvious that the shoutcast server was basically locking up all the performance, but the "canned response" in words is just stupid. I commend the actions but the BS that "blame the sim owner" is just so stupid it is funny.

* * * * * * * * * *

Foxyfurman:

Re: Huge Lagg issues on my sim

Spontaneously about a week ago we have had HUGE lagg spikes, we have tried shutting down scripts section by section, with zero help, your service desk told me to re compile all my top scripts into LSL, this is NOT the problem, and it will break many items. We have shut down section by section we still have huge problems, basically my sim is un-useable at this time. Again I have tried shutting scripts off section by section and the only way it even works OK is when ALL scripts are off, even a few scripts will over tax the sim. I have NEVER in a year had this issue and have added nothing new, yes I have restarted, many times including a LL initiated re start. I was informed by my manager that this could have started mid Jan.(ish)but just not as bad, not sure who is new sharing our server but this is a huge mess, thanks for taking care of this ASAP.

* * * * * * * * * *

Gareth
Linden Lab Support:

Hi Foxyfurman,

Thanks for contacting us in regards to this issue.

I am going to escalate this ticket to the appropriate department and they will contact you shortly via email regarding your request. In the meantime, if you need to contact us again about this issue, please add a comment to this ticket rather than creating a new ticket.

* * * * * * * * * *

Spike Linden
Linden Lab Support

Hello Foxyfurman,

Thank you for your ticket regarding your region and the performance issues.
I'm sorry to say that if there are lag spikes in the way you have described and they continue after restarts, then the source of the problem is within your region. As you have scripted spikes, then it will be scripted items within your region causing this.

Every time you restart a region, it will change server, so other regions can not keep causing issues if you move from host to host.

How were you disabling scripts section by section? This can't actually be done completely.
Unchecking the Run Scripts option in About Land will only disable scripts for non-parcel owner. You can't actually disable scripts for items that are owned by the parcel owner. So if a parcel is deeded to a group, all deeded items will still be running. If a parcel is owned by an avatar, then all items belonging to that avatar will still be running, with both 'All Residents' and 'Group' unchecked for Run Scripts.

This can help you narrow down the problem though, as you can eliminate most parcel/objects through this method.

I've had a look at the region today and script time is high (18-20ms). The chances are high that as avatars go to the region you will be running low on available resources and start to have issues. In this case your 'only' option is to look at your content. You will have to either re-script more efficiently or remove items completely from the region. There is no other choice for this type of issue.

Please make sure to look at our wiki page for 'Improving Region Performance'. This page can help you greatly in managing your region better. Key thing to remember is that you can't just fill a region with anything you want and expect it to work. You 'must' manage your regions content effectively.
TICKET CLOSED

* * * * * * * * * *

Ticket RE OPENED

Foxyfurman:

This was fixed prior to your investigation :)

Someone from LL moved the shout cast server sim that was running on the same CPU and like magic everything is totally fine. We are crazy careful about what we put down and paranoid about Time Dilation and our total script time. With a shout cast sim as a "virtual neighbor" our time dilation was 70 now it is 99. In addition for fun we ported to these shout cast sims and all of them were horrific with lag/with time dilation in the 80-50 range and all of their virtual neighbors were a train wrecks as well regardless of ANY avatars.

We had shut down all scripts in the estate manager panel and still had lag this is how we know to look OUTSIDE our sim. It is not always the sim owners fault as you say.

Sorry I re open this ticket but this caused my sim to basically stop in its tracks, it will never happen to me again because I will know what to look for, but I know others will not be so creative in looking outside their sim for the problem.

* * * * * * * * * *

It seems persistence is in order when dealing with the Lindens. Fortunately, Foxyfurman has plenty of it.

Bixyl Shuftan

A Laggy Problem at Foxworth

Recently, the lag at my home sim at Foxworth became particuarly bad, just after January 20. Sim owner Foxyfurman Kumani was among the first to notice, saying he found he was barely moving, and the only others in the sim at the time were five in the Mayan Casino, “It was not long after a Linden Lab restart of the sim.”

Explaining the problem in a group notice, “The first two calls I had to Linden Lab were a total waste of time. One suggestion (of theirs) was to restart the sim. (laughter) Okay, this did not work. The other was to look at our top scripts. We already did this, and even returned new scripts that had been put out during the period of time when the problem started. We tried turning off scripts section by section (in the sim) with no positive results.”

Both his sim manager Aiko Swashbuckler and the sim technician Leonarris Orsini helped Foxyfurman in looking for the problem. Not having found anything, Aiko asked everyone living in the sim to stop tinkering with scripts. But as it turned out, none of the tenants was behind the problem either, “no one in Foxworth did anything to cause this.”

So what caused the intermittent lag? Leonarris did some investigating and finally found the source on Friday Jan 29. The problem was the way sims are stored within Linden Lab’s servers. A sim doesn’t have to be physically next to others on the map and Grid to cause problems. If two sims next to another in storage, what happens in one can affect the other. Aiko explained Foxworth was one of “ several sims on one server” with “three Class Five ones. One of our ‘neighbors’ on the same (server) was running a Shoutcast server farm business.” Foxy went on, “a new place just came online a week or so ago. ... It is so laggy that you can’t even walk at that place, and it is sucking resources from the machine. ... Linden Labs knows of the problem and will fix the recourse allocation or move us to another (server).” He stated Foxworth’s place on the map would not be changed, at most it would “just run on a different machine.”

The following day, Foxy made another group notice, “Linden Lab has moved the offending sim to another server. if you have noticed, you can move now and walk and fly, all good things. Thanks to everyone who helped, and those who suffered the lag along with us.” And everyone was in complete agreement, “it was a nightmare,” “yeah that Shoutcast thing was crap, I'm glad they took care of it so fast,” “Indeed, the lag was horrid. O.o “

And so, peace and harmony was restored to our little corner of the Metaverse.

Bixyl Shuftan

More Trouble at Foxworth

Earlier, the Foxworthy sim was the scene of trouble when a number of accounts were permanently banned for reasons that made no sense to anyone familiar with the area. More recently, the region was the target of a griefer, whom went as far as to threaten someone. How the Lindens reacted was a surprise.

To read more, go to Community.

“Linden Labs, Why the Hell Attack Me?”

The Foxworth sim, owned by Foxyfurman Kumaki, gets some residents passing through, notably at the Mayan Casino. But compared to the big gaming areas in Second Life, it hardly ranks in visitor numbers as a high-traffic place.

So why have there been some bannings by Linden Labs for “gaming” traffic numbers in the sim?

“Feels like I am (being) attacked,” Foxyfurman told SL Newspaper, “Look at Search, then the places tab. ... Type in games or Zyngo. Look at the top listings. Those are the places with the highest traffic, like 50K, 60K. Our sim the day after the fishing (tournament), was 15K. To get 50K, you stack ‘bots on each other until your number gets up. We don’t do that. Our numbers are normal. Can you imagine, you can hardly move (around) with the fishing event. To get that traffic all the time will close the sim with lag.”

“That us why twenty to sixty thousand is fake, and every one knows it. That is why when they say I am doing this, why the hell attack me?”

“Now you look at a list of discipline from Linden Labs. No one ever gets banned forever, just a warning or suspension. John (Diddy) got no warning, and is perma-banned for life. ... I just wish they removed the ‘places’ from Search. That way, no one would worry about bots, or sitting in your own sim. (laughs) I’ll get banned for listening to my own stream. If I was idle with Diddy, I would have been. Thankfully, I was not.”

Foxyfurman had a notecard, which contained his responses to the Lindens, plus a comment at the end.


Details: Friends and a few alts banned from SL for gaming traffic.

First I want to say I am disappointed and concerned, why a resident that has 2 mainland sims and 1 private region would be singled out for gaming traffic, when the traffic numbers are not high we are normally between 2+K to 12K all legitimate from Halloween hunts, 7 seas fishing events, and Midnight mania boards, this traffic is not high and the #s totally support my claim. We do not in any way rely on traffic to build our sim, for that we would need an army of bots and I don’t want to bother with that kind of silliness.

My real life business partner was working on a platform above our sim, (Diddy Darkmatter), He has a few of my alts out so he can test a listener device he has been working on with another resident, (Aiko Swashbuckler), then poof!! Everyone is banned from SL!

1. I want everyone who was banned on my sim reinstated (this may include customers I don’t know about who were banned as well).
2. I want and explanation of WHY my sim was a target of this attack, yes I call it an attack because our traffic is obviously not generated by bots; it is too low for that.
3. This is a fun place for me, I have the music playing on speakers in my warehouse all day while I work, and can I expect to be banned for sitting in my own sim listening to my music? I am really worried about this, and to be honest taking the time to write this letter is costing me time and stress that is totally unwarranted and unneeded.

In the end I want to say I am not mad in anyway, I am confused and a bit disappointed that you would treat a customer like this, it feels like a vendetta and while not a paranoid individual I still ask for a detailed account of the decision process as well as to speak to the individual(s) who were involved in this decision process, I hope the individuals involved do not have access to my billing information as well, as a business owner RL taking someone’s billing information and a customer/business relationship is to be taken very seriously and I hope this will be handled in the best manner.

I wish to talk to a real person for this as well my cell is listed below.

Thank you for your consideration

***-***-****
Foxyfurman Kumaki


Perry Linden Hi FoxyFurman,

Thank you for contacting us.

Your ticket has been escalated to the correct department.

They will contact you as soon as possible.

Kind Regards
Perry Linden


3/11/2009 4:05 AM PDT Closed
Matthew Linden Hello,
Linden Lab just completed its review, at your request, of a disciplinary action taken against your account.

Our investigation shows that the suspension of your account was correctly applied in alignment with the Second Life Terms of Service and/or Second Life Community Standards.

If any further information is required, Linden Lab will contact you regarding this incident. Otherwise, please consider the matter resolved as no further communications will be sent.

Sincerely,
Customer Support
Linden Lab


3/11/2009 4:56 AM PDT Reopened Foxyfurman Kumaki Dear Support,

You have given me answers to none of my queries.

You discipline a long time customer and sim owner and they maintain and still maintain innocence. I ask you for not a generic answer but details of WHY this was done; the canned response I get is not acceptable. You have also banned a paid account (Diddy Darkmatter) this is my business partner and he needs his account restored. He will contact you from his air card if he can as he is out of town ATM. Also you may have banned some customers as well, when you handed down your discipline a few other customers dropped off of radar as well. Again I pay US 330ish a month to LL, I just want a detailed account of the discipline not a canned response.

I want a call from a real person again I ask:

Thank you for your consideration

Foxyfurman Kumaki


Ticket closed and locked by support with no response Actions
You are not authorized to perform any actions at this time.

*********
In the end, my partners account was permanently suspended and as well my 2 alts I use for testing perms and games. I am not sure how many customers got the boot but I know some dropped off radar at the time of the LL raid.


Later, another notecard was sent from Aiko Swashbuckler, with some links to the Linen blog concerning rules about bots.

https://blogs.secondlife.com/message/9040#9040 - Further Clarification on Bots and Camping, “So the policy statement is that where we see a Resident unfairly increasing their Search ranking, regardless of how that is achieved, it will be considered as 'gaming'. We will give a first warning to begin with and direct people to the policy. However continued gaming can result in suspension or removal from Search listings altogether.”

https://blogs.secondlife.com/message/28188#28188 - Re: Camping Crackdown, “I have heard that all residents on the sim were suspended for the hour, many may not have known and will be surprised when they see the email explaining why they were suspended. Did a Linden go rogue on us? Did the rules change while we weren't looking? I believe that suspending the campers is NOT disciplinary action.”

http://wiki.secondlife.com/wiki/Linden_Lab_Official:Inworld_policy_on_bots - Official Policy on Bots

Following this were several examples of disclipinary action taken by Linden Labs on related matters:

Date: Sunday, November 8, 2009
Violation: Community Standards : Gaming Traffic Violation
Region: Ruwart
Description: Gaming Traffic.
Action taken: Suspended 1 days.

Date: Thursday, November 5, 2009
Violation: Community Standards : Gaming Traffic Violation
Region: Wexler Education
Description: Gaming Traffic.
Action taken: Warning issued.

Date: Tuesday, November 3, 2009
Violation: Terms of Service: Gambling
Region: Acris
Description: Terms of Service.
Action taken: Warning issued.

Date: Tuesday, November 3, 2009
Violation: Community Standards : Gaming Traffic Violation
Region: Dozer
Description: Gaming Traffic.
Action taken: Warning issued.

Foxyfurman has heard no further word from the Lindens about his friend’s account, but unfortunately the next message by the lindens concerning the bannings was not a good one, “fact is this same thing happened to my SLX account.”

“This is one of those things that it’s best that everyone knows what happened and exactly how Linden Labs handled it.”

Bixyl Shuftan