Friend advice please, what would you do in my shoes? Please read!?

Hi. I have a group of three close friends. The problem is, although our official friendship group consists of us four, we always have a lot of 'tag-alongs' who always follow us around in school. There are always at least three other people hanging around us constantly, at lunchtimes seven other people come and join us and there's often up to nine other people tagging around after us! I don't really mind them, some of them are quite nice, and noones really horrible or anything, so I don't mind TOO much, but the result of this is that I never speak to my three friends alone! I mean sure, if we have to work in pairs, it'll nearly always be us four pairing off rather than thr tagalongs, unless one of them pounces on you before you can reach your main friends and you have to say yes, but I never really see my friends anymore without five or six people trailing after us. So last week I organised a trip that me and my friend (I'll call my friends E, Y and L) Y and I do quite a lot. It's a trip that is very personal to me, and whilst it's not exactly private, I consider it an important trip. I decided I'd ask E along too, but i'd prefered not to ask L because I knew she couldn't take the trip seriously enough. Sadly E and Y had to cancel, so this afternoon at school Id briefly mentioned that we'd reschedule the trip to this week, but no further details were discussed, and E and Y agreed. L had been hanging around next to us, but she misheard as she wasn't really listening and thought I was talking about our Wednesday singing group, to which she said she couldn't make it. I knew she had misheard but I didn't want to put her right as although I do like her abd she is one of my close friends, I'd really prefer it if she didn't come to this particular occasion. However she just texted me and asked if me, her, E and Y wanted to go to town this weekend because she said we hadn't done anything together in a while. It's true, but I really want to go on my trip! The details for the trip hadn't been officially aranged and I only briefly mentioned it to E and Y today as I knew we had the rest of the week to arrange it if nessecary. I don't like the town much and my trip does mean a lot to me for personal reasons. It's not the trip of the centuary, but it wouldve been nice to go with E and Y for a bit. But if I turn L down I'll feel horribly selfish, and I'm not sure if E and Y will remeber we have arrangements this weekend as we only said a few sentences on it this morning, so if they accept and I decline I'll miss out entirely! I mean it wouldn't kill me to go to town, and it wouldn't kill me to ask L along on my trip, but I'd much prefer not to. What would you do in my shoes? Thanks.