With minor variation, pretty much every job search for an academic tenure-track job works the same way:
1)Post an ad describing the type of research you are interested in adding to the department.
2) Wade through the resulting applications looking for metrics of success (publications, grant funding, awards)
3) (optional) Phone interview a subset of candidates (~10).
4) Invite 3-5 candidates for an on campus interview, during which they will meet with department members one-on-one, give a seminar about the work they have done, give some presentation on work they would do at the interviewing institution, eat a bunch of meals with the search committee.
From a logistical standpoint, getting the campus interviewee list right is pretty critical. If a committee goes through the campus interviews and doesn't find a suitable candidate, the search is either delayed while they scramble to bring different people in or is a failed search that may or may not re-appear in the following season. Neither option is good, so the committee really wants to get those invites right.
However, the metrics of success we use to evaluate people on paper don't always translate into strong interviews. It seems that every search has at least one interviewee who self-immolates in some way. I have been involved in searches in which only one candidate turned out to be acceptable to the committee.
This has led myself, and several others I have talked with informally, to wonder whether there is a better screening process by which we could identify candidates. There are so many good scientists out there who just need a shot at an interview, but can't get one.
It's an imperfect process, and yet I have never seen a department try anything different. What could we do better?