Quoting movie_madness:

Quote:
DVD Profiler is nice and all but if you're like me, you also have laserdiscs, VHS, Video CDs, DVR recordings, and online movie downloads, and DVD Profiler can't include them.  I have a considerable amount of movies not on either DVD or Blu-ray.  And I have tons of TV shows that will never ever be on any home video format at all (Oscar shows, rare TCM movies, and the likes).  In my case, DVD Profiler just seems woefully inadequate to catalog my collection.  So I decided to take matters into my own hands.  It had to involve IMDb, since it is the only possible place for me get info on any title conceivable.

If you have programming skills then you must have thought of doing something like this, or may have already done it.  First, I installed the Python programming language.  Then I installed IMDbPY, a set of Python scripts that can retrieve data from IMDb.  I modified the scripts to suit my particular needs.  I created a text file of all my movie titles in my collection.  I imported the file to a Python script, and got a list of IMDb movie IDs (e.g. 0157059) in return.  I used the movie IDs and ran another script, and got a list of cast and crew of all those movies.  I transferred all the IMDb data I got to my choice of database program (in my case, Microsoft Access).  Lastly, I wrote a few vbscript web pages that connects to the Access database.  The webpages let me search for names and return the info I want.

I had thought this project would take me ages, but it didn't!  I have 5000+ movies and TV shows in my collection and it only took me a week and half to get the necessary IMDb data for about 4000 shows into my database.  And the process was not entirely automatic either - I had to look up IMDb movie ID one by one.  But IMDbPY performed pretty speedily.  It took only 1-2 seconds to look up an ID.  It took just less than an hour to retrieve the complete cast and crew of a THOUSAND movies.

Of course, I had looked for third-party software that can import IMDb data but none worked as I had wanted, mostly because they retrieved incomplete data.  With IMDbPY, I was able to retrieve COMPLETE cast and crew, complete everything, just like you see them on the website.

Has anyone here actually done this??  Now I'm able to say that I have.

Nice work!

Sounds pretty sweet. 

Much easier and better way would have been using Custom Media type and plugin that imports Cast & Crew data from IMDB. IMDB on the other hand doesn't have episode level Cast & Crew info for TV-series'. Only disadvantage would have been that if you have several copies of the same film, editing Cast & Crew woud require some manual copying.

IMDb does have episode-level cast & crew, but it doesn't have *season*-level cast & crew, which I actually need for my project.

Quoting DJ Doena:

Quote:
Quoting movie_madness:

Quote:
IMDb does have episode-level cast & crew, but it doesn't have *season*-level cast & crew, which I actually need for my project.

Simple: Get all the episode infos from IMDb and check them. If a person has appeared in (let's say) >= 90% of all episodes, s/he is a season cast. Everyone else is episode cast.

You don't want to make assumptions like that.  Some actors may have appeared in early seasons and left the show later; some may have joined the show later.  Not to mention, TV shows with no regulars: anthology series (Twilight Zone), documentary series, and the likes.

You just have to do it the hard way and create one big table with complete season, episode, cast info:

Season  Episode  Cast
1          1        Actor 1
1          1        Actor 2
1          1        Actor 3
1          2        Actor 1
1          2        Actor 3
1          2        Actor 4
2          1        Actor 1
2          1        Actor 2
2          1        Actor 4
2          2        Actor 1
2          2        Actor 4
etc.


Then create a season-level summation:

Season  Cast
1          Actor 1 (2 episodes)
1          Actor 2 (1 episode)
1          Actor 3 (2 episodes)
1          Actor 4 (1 episode)
2          Actor 1 (2 episodes)
2          Actor 2 (1 episode)
2          Actor 4 (2 episode)

Quoting Winston Smith:

Quote:
As Kino said you are not the first or the last to clone IMDb. The single most inaccurate database in the World, for the reasons Kino alluded to. They have or had no standards for data entry.

This is supposedly the selling point of DVDP but I'm not so sure about that.  Yes, DVDP's database is handled by people who are more serious about managing a database than IMDb users.  But there are FAR FAR fewer people doing this for DVDP than the people who do it for IMDb.  If there are 10,000 people posting entries to a movie on IMDb, the chances are VERY good that we end up with very few errors -- even if 1000 or 3000 of those 10000 people are idiots who post entries that are way off.  This is because the sheer number of times this movie is being looked at and worked on would make it highly likely that a very high percentage of errors would be corrected when it's said and done.  I think this is the reason why I rarely see errors in the IMDb's cast and crew.  I may see *incomplete* cast and crew.  But I rarely see outright errors.  One outright error I notice is the name of the actress who plays Josef K's cousin in Orson Welles' "The Trial".  IMDb has her name as Naydra Shore,  when on-screen credits show "Maydra Shore" as the name.  I have a copy of the movie script and it also shows "Maydra Shore".  This shows how obscure an entry has to be to produce an outright error.

Just want to digress a little bit on how a very high number of half-reliable data would often excel over a much lower number of more reliable data...

People used to say speech recognition and handwriting recognition couldn't be done.  They said if we humans couldn't even recognize somebody's handwriting or speech half the time ourselves, then how could a computer?  It turns out a computer has one capability humans don't have: the ability to accept and process a ginormous amount of data.  Let's say we want a computer to recognize the handwriting of the letter "L".  What do we do?  We feed thousands and thousands of handwriting samples of the "L" into the computer.  Some people write the "L" with a loop, some L's are slanted left and some right, some L's are just a vertical line, some are printed, etc. etc., plus every other conceivable way of writing an L.  We feed all these into the computer, and with a good algorithm, it CAN determine if ANY piece of handwriting is an L or not with a VERY HIGH degree of certainly. I think anyone who has used a smart device's handwriting feature would notice this.  Plus, the growing speed of computer processors (CPUs) over the years have helped to make this goal a reality.  So there is why the SHEER number of samples, such as those possible on IMDb, *CAN* often transcend themselves into something previous thought not possible.  To look at it in extreme closeup: let's say IMDb user #1 posts in the quote section that Bogart says "Play it again, Sam" in the movie "Casablanca."  User #2 amends it with "Play it again."  User #3 thinks, no, he does say "Play it again, Sam".  User #4 posts "Play it, Sam."  And so on and so forth.  Now what is the probablity that by the 10,000 poster, that we would have the correct quote posted?  Very high.

Quoting movie_madness:

Quote:
DVDP's database has no info whatsoever on the *bonus features* of the discs.  Some bonus features are notable documentaries, featurettes, interviews, etc., and they have cast & crew too, and they certainly deserve to be catalogued.

Quote:
DVDP's database does not catalog the participants of audio commentaries, or any audio-based bonus features.  Wouldn't it be nice to be able to search what audio supplements someone has done?

Whether or not something deserves to be catalogued or is 'nice' is in the eye of the beholder.  Speaking for myself, I can live without either of those...especially audio commentaries as I never listen to them.

First of all, nicely done.  I wouldn't do it to my database, but we're not talking about mine.  So well done there.

Second of all, as long as you don't try to contribute this data, you're just fine.  It's YOUR database.  You should set it up in a manner that works best for you.  If IMDB data is what you want, then you're in great shape.

Third, next time you should start here.  Your Python solution sounds very clever and was probably the best route in this particular case.  But we do have some great user-created tools that you could possibly use to your advantage next time around.  We can also assist with other things too that may save you time if you're not as familiar with the program (though it sounds like you have a solid grasp at this point!)

Finally, IMDB is on-par with Satan for most users here.  Trying to convince them that IMDB data doesn't come from the pits of Hell is a pointless exercise.  But hey, if you like banging your head against the wall, go for it!