| Author |
Message |
Nathan Robinson
myArmoury Admin
|
 Posted: Thu 20 Jul, 2017 11:49 pm Post subject: FIXED: Photobucket image hosting |
 |
|
What is Photobucket and why does myArmoury.com need a fix?
Many people wishing to share images on the 'net use the popular image hosting site Photobucket to host images that can then be posted in topics on forums like myArmoury.com. They provided this service free of charge, and for a little bit of cash would allow their users a bunch of bandwidth as well. Recently they changed their business model and now want to charge members $400/annually to allow for their photos to be linked and shown on other web sites.
This decision breaks millions of forum topics all over the Internet, eBay auctions, and other pages that have had images from Photobucket on them, showing this image instead of the one intended:
You can read more about the problem here
This was particularly upsetting to me given that I own this site and suddenly in one fell swoop hundreds of topics here were suddenly shown with broken images! We've had some great things posted here so I had to do something.
Can't something be done to fix this?
With a bit of programming magic I came up with a solution that now correctly displays the Photobucket hosted images in our topics!
Check out this topic here as an example
That topic contains a lot of images hosted on Photobucket that would show up as broken until my newly implemented fix.
So what?
Topics are hard to read when they're full of broken images. A lot of our really great discussions here are image-intensive and really rely on those images to tell the whole story. Without them, a significant percentage of our content's value is diminished.
I think myArmoury.com might be the only forum to have even tried to tackle this problem at this time. I believe I'd bet I'm the first to come up with a solution. I won't reveal how I made this fix but will say that my solution is pretty creative if I say so myself.
This fix is still being tested so there are bound to be some issues with it.
Can I see some examples?
All the photos below are hosted on Photobucket. Without the fix, they would all show up as broken images. They're displaying correctly now (assuming that the people don't delete them from their Photobucket accounts, of course!)
.:. Visit my Collection Gallery :: View my Reading List :: View my Wish List :: See Pages I Like :: Find me on Facebook .:.
|
|
    |
 |
Greyson Brown
|
| Posted: Fri 21 Jul, 2017 7:46 am Post subject: |
|
|
$400 annually !?! That is highway robbery. You could pay for actual site hosting and upload pictures for the purpose of linking so much cheaper (maybe even free depending on the WordPress.com terms of use).
You said you won't share how you worked around this problem so I won't actually ask that, but I will say that it obviously took some research and hard work. Adding a database trigger to update the links is the obvious part, but figuring out what the link needs to be is still eluding me. I don't really have a need to know how this was done, but I may waste part of the day trying to figure it out anyway.
Thanks for taking the time to figure this out. You are right that many topics on this site lose a lot of value without working images.
-- Greyson
"So long as I can keep the path of honor I am well content."
-Sir Arthur Conan Doyle, The White Company
|
|
|
|
|
|
Stephen Curtin
|
| Posted: Fri 21 Jul, 2017 8:28 am Post subject: |
|
|
As always Nathan your efforts are much appreciated.
Éirinn go Brách
|
|
|
|
|
Mikko Kuusirati
|
| Posted: Fri 21 Jul, 2017 11:24 am Post subject: |
|
|
Applause, sir! Uproarious applause!
I moved to Imgur as soon as Photobucket started spamming me with honest-to-gods extortion emails some weeks ago ("Nice photos you've shared, be a real shame if anything happened to them," and embarrassingly close to those actual words) but of course that doesn't help all the old threads on all the forums I've posted images in and can't edit anymore...
I'm fairly impressed and quite curious how you did that. But if you don't wanna spill, eh, fair I guess. 
"And sin, young man, is when you treat people like things. Including yourself. That's what sin is."
— Terry Pratchett, Carpe Jugulum
|
|
|
|
|
|
Daniel Sullivan
|
| Posted: Fri 21 Jul, 2017 12:58 pm Post subject: FIXED Photobucket image hosting |
|
|
Nathan,
Wonderful! Frankly, I was getting a bit depressed over this situation as I believed the site had suffered a serious wound. Now I believe in "miracles" and suspect there might even be a "higher authority" .....
A well earned Thank You!
Regards.
Dan
|
|
|
|
|
Luka Borscak
|
| Posted: Fri 21 Jul, 2017 1:05 pm Post subject: |
|
|
|
I checked my latest review and all pictures are there it seems. I don't know how you did it, but thank you.
|
|
|
|
|
Nathan Robinson
myArmoury Admin
|
|
|
|
|
Greyson Brown
|
| Posted: Fri 21 Jul, 2017 3:24 pm Post subject: |
|
|
Nathan,
My hat is off to you for finding a work around for this without throwing computers out the window. I have spent a total of maybe 3 hours looking into this today and I can feel my blood pressure rising. I created a junk photobucket account and uploaded a picture. I could see my picture and link to it from an external site for about 20 minutes; after that I couldn't even view my own picture when logged into photobucket. From within their own site, I got the "update your account to allow 3rd party hosting" message! They seem to not understand the concept of "3rd party" (or the difference between update and upgrade).
You, sir, have my respect for being able to tolerate their festering, ad-filled excuse for a site long enough to solve this problem.
-- Greyson
"So long as I can keep the path of honor I am well content."
-Sir Arthur Conan Doyle, The White Company
|
|
|
|
|
Aaron Hoard
|
| Posted: Fri 21 Jul, 2017 5:13 pm Post subject: |
|
|
|
Thank you for working to resolve this.
|
|
|
|
|
|
Lin Robinson
|
| Posted: Sat 22 Jul, 2017 6:30 am Post subject: |
|
|
| Nathan Robinson wrote: | | Luka Borscak wrote: | | I checked my latest review and all pictures are there it seems. I don't know how you did it, but thank you. |
Occasionally, when I'm not being foolish, I can be pretty clever.
Your recently posted-review was one of the main reasons I put the time in to figure out a hack to fix this problem here. |
I will echo what the others said and extend my thanks to you for the effort expended and sympathize about the angst you undoubtedly suffered when this happened. I have used Photobucket from time to time on this and other forums but I don't think I will any more. I just went to my account to see if my photos were still there and they are, but I have them elsewhere as well. What I did notice were a large number of ads inserted in my catalog of photos and an offer to eliminate the ads for $2.49 per month. Not taking that offer either. IMHO many of these companies, once they get their infrastructure and staff in place, don't have a lot of other expense and ploys like this are simply to increase their profits on the backs of folks that they think must use their services. Wrong about that, at least in my case. I can live without them.
Lin Robinson
"The best thing in life is to crush your enemies, see them driven before you and hear the lamentation of their women." Conan the Barbarian, 1982
|
|
|
|
|
Nathan Robinson
myArmoury Admin
|
| Posted: Sat 22 Jul, 2017 4:36 pm Post subject: |
|
|
Hi Everybody-
I did some statistical analysis of the Photobucket images that have been posted on our forum:
Number of PB images posted: 62,233
Distinct number of PB images posted: 56,481
Since the fix we have already served up 12,228 distinct images at this time. These images have been cached locally in case PB changes their technology, moves the images, or goes out of business. This number will just keep growing as posts are accessed.
.:. Visit my Collection Gallery :: View my Reading List :: View my Wish List :: See Pages I Like :: Find me on Facebook .:.
|
|
|
|
|
|
Jonathan Hopkins
|
| Posted: Sat 22 Jul, 2017 6:56 pm Post subject: |
|
|
|
Well done, and thank you, Nathan!
|
|
|
|
|
Nathan Robinson
myArmoury Admin
|
|
|
|
|
Nathan Robinson
myArmoury Admin
|
|
|
|
|
Greyson Brown
|
| Posted: Fri 18 Aug, 2017 6:36 am Post subject: |
|
|
| Nathan Robinson wrote: | Since the fix we have saved 39,431 distinct images from Photobucket's prison.
These images have been cached locally in case PB changes their technology, moves the images, or goes out of business. This number will just keep growing as posts are accessed.
This is a whole lot of content we saved from the ashes. |
Perhaps a pointless suggestion, but have you considered querying the database to pull all those post that have photobucket images and then putting those URLs in a script to open those pages? That way you could ensure that all of the images are backed up without waiting for people to manually access those posts.
-- Greyson
"So long as I can keep the path of honor I am well content."
-Sir Arthur Conan Doyle, The White Company
|
|
|
|
|
Julien M
|
| Posted: Fri 18 Aug, 2017 9:53 am Post subject: |
|
|
I noticed the pics were live again and naively thought Photobucket had second thoughts about their rather unfriendly conduct...but could not find anything to corroborate that hypothesis online - I can see why now
Well played Nathan - would have been a shame to loose all that content!
I'll stick with Imgur from now on...
J
|
|
|
|
|
Nathan Robinson
myArmoury Admin
|
| Posted: Sun 20 Aug, 2017 7:00 pm Post subject: |
|
|
| Greyson Brown wrote: | | have you considered querying the database to pull all those post that have photobucket images and then putting those URLs in a script to open those pages? That way you could ensure that all of the images are backed up without waiting for people to manually access those posts. |
I have considered it but could not think of a good way to make this happen. Instead, I relied on the fact that web crawlers by Google, Yahoo, etc, index our site daily. I can log into my Google search console and request a reindex with a supplied sitemap. I created a sitemap with all the topics and requested the reindex. This essentially does the same thing as you're proposing.
.:. Visit my Collection Gallery :: View my Reading List :: View my Wish List :: See Pages I Like :: Find me on Facebook .:.
|
|
|
|
|
Greyson Brown
|
| Posted: Mon 21 Aug, 2017 7:21 am Post subject: |
|
|
| Nathan Robinson wrote: | | Greyson Brown wrote: | | have you considered querying the database to pull all those post that have photobucket images and then putting those URLs in a script to open those pages? That way you could ensure that all of the images are backed up without waiting for people to manually access those posts. |
I have considered it but could not think of a good way to make this happen. Instead, I relied on the fact that web crawlers by Google, Yahoo, etc, index our site daily. I can log into my Google search console and request a reindex with a supplied sitemap. I created a sitemap with all the topics and requested the reindex. This essentially does the same thing as you're proposing. |
I would do it by using a query that builds the commands needed for a script. I use Linux almost exclusively, so I would use:
SELECT concat("firefox http://myArmoury.com/talk/viewtopic.php?t=",topicid," &")
FROM posts_table
WHERE post_content LIKE '%photobucket%';
(replace firefox with browser of choice, and, of course, provide correct column and table names).
I would then create a script (just a text file with permission to execute it) that contains the output from the above query (just copy and paste) followed by:
sleep 1m # wait a little bit just to ensure the pages have time to load
pkill -f firefox # close firefox when done
Run the script while you eat lunch or something where the computer is not actively in use as all of those browser windows opening would get annoying if you are trying to work.
It has been a while since I have done Windows batch files, but I think the only changes are that the & at the end of each line is not needed, sleep is not supported (the work around is to ping 127.0.0.1 60 times (each ping is close to 1 second)), and pkill is replaced with taskkill.
Your solution works just fine, though.
"So long as I can keep the path of honor I am well content."
-Sir Arthur Conan Doyle, The White Company
|
|
|
|
|
Greyson Brown
|
|
|
|
|
Nathan Robinson
myArmoury Admin
|
| Posted: Thu 24 Aug, 2017 4:32 pm Post subject: |
|
|
Hey Greyson, I don't want to clutter this topic with technical stuff other than to say this: I know how to make a script that queries the pages, but there's a lot of other complications such as topic pagination (digging down into multi-page topics), passing session variables/cookies, server load, and some other database-specific stuff I don't want to mention here. At this point, it's moot because...
~~~~~~~~
Since the fix we have saved 51,228 distinct images from Photobucket's prison.
These images have been cached locally in case PB changes their technology, moves the images, or goes out of business. This number will just keep growing as posts are accessed.
We've saved 90% of the 56,481 hosted images.
.:. Visit my Collection Gallery :: View my Reading List :: View my Wish List :: See Pages I Like :: Find me on Facebook .:.
|
|
|
|
|
|
|
This is a standard topic