How to judge the Leading Lights

Here's a quick guide for judging the Leading Lights entries.

Helpful links

Log in

Make sure you can log into the Leading Lights Judging Portal. 

Set your password by entering your email address as the username and selecting "lost password." Then follow the password reset instructions.

The judging process

Once you log in, you'll see the categories and number of entries you've been assigned to score. The finalists have already been announced, and there will be 10 or fewer entries in each category from which you'll pick a winner.

Our ranking system

Light Reading uses a 1-3 ranking system, with 3 being the highest score possible and that's the entry you're proposing as the winner.

If you have one or two runner-ups, please score those with "2." Everything else that doesn't make the podium gets a "1."

The scoring process

Please read each entry carefully, review the category descriptions and look at any supporting documents companies have linked to their entries.

Once you score each entry, please write a sentence or two in the comments section so we'll know why you scored it the way you did. 

Your comments and scores are not shared with anyone except our editors. We don't even tell our audience or entrants which judge is assigned to which category. 

Your comments are incredibly helpful to me in those rare cases when we need a tie-breaker. They also help us know what stood out or what the entry lacked. If judges tell us what info they wish they had to round out each entry's "story," we can give better guidance for future contests.

The entire judging process is online, so you can only save your work as you go. There is nothing to store on your computer or print out. 

The system requires both a score and a comment (placeholders are fine) before it allows you to save your comments.

Look at those links

Please consider the links submitted by each finalist as part of the entry. Some links will take you to folders with loads of stats, details, videos and other info that can tell a much better story about the technology, person or use case you're evaluating. Don't skip over the links – check them out.

Staying organized: Application numbers

Once you click "open" from your judging assignment page, you'll notice that the list of entries may not be helpful, as it shows information about who submitted the entry (usually PR firms and agencies).

However, each Leading Lights finalist entry has a unique application number, so I recommend writing those down separately along with your own notes as you read through the material.

Write down the application numbers separately so you can track which entries you've read and what companies submitted them.

Keeping your own notes might help you, too, if you get interrupted a lot like I do.

Here's how I handle my judging assignments

Once I write down which entry (application) I'm working on, I read it carefully. 

At first, I score all my entries with "1" as a placeholder. I add the day's date as a placeholder in the comments section and hit save once I've read the entry all the way through. This helps give me a way to find out where I've left off in case I'm interrupted. Also, I don't want to give any final scores until I've read each entry in the category.

While I'm working through the category, I make notes on what stands out or what info I'm missing. I'll summarize that and add it to the comments section later.

After a break of a few hours (or days), I return and reread the entries that best lived up to the category description (and any clarifying remarks made in the awards FAQ). 

Once I have a feel for how each entry compares to its finalist peers, I can confidently adjust the scores (one entry gets a "3," and a couple of runners-up will get "2s"). Then I condense and clean up my comments, add those to the entries and save my work. 

Wrapping up

 Once you've both scored and entered a comment for each entry in your category, you're done with that category. 

At that point, please send me an email with the category name and #LL24 in the subject line and the word "DONE" in the body of the email.

Thank you for being an incredibly important part of Light Reading's 20th annual Leading Lights Awards. We couldn't do this without you!