I've been loading for the 3" Charter Bulldog for many years, and have 2000 rounds through one specimen, which is running just fine. The key here is to select loads appropriate for a safe, but very lightweight revolver. The "Skeeter" load with 250 grain Keith bullet is frankly not that. Even Skeeter Skeleton wrote that he reduced his load from 7.5 to 6.5 grains of Unique in the Bulldog because of the recoil abuse to both the pistol and shooter. M.L. MacPherson did an excellent article on this topic some years ago, "Care and Feeding of the Charter Arms Bulldog", if you can locate it.
The typical 240-250 grain cast bullets generally give poor accuracy in the Bulldog, even at lower speeds, as they just don't stabilize. The standard 240 grain soft swaged lead bullets shoot better, but have to held below about 700 fps. If you are a fan of that kind of ballistics, have at it.
I've found the best results from a standpoint of cast bullet accuracy is in the range 165-200 grain bullets. Rimrock makes a 200 grain sharp wadcutter which gives very good results launched anywhere from 700 to 850 fps. This might be a good defensive bullet as well. My usual practice loads are 200 grain cast RNFP bullets at 750-800 fps. I've used 165 grain TC bullets up to 850 fps and these are also quite accurate. The advantage of these loads for practice are obvious: less abuse of shooter and firearm, with good accuracy. I have used either 700-X or Titegroup mostly for these light bullet loads, but Win 231/HP-38 is also a good choice. With a 200 grain cast bullet 5.0 grains of Win 231 in the Bulldog will average 750 fps +/- 25 fps. There is plenty of suitable mainstream published loading data available for these types of loads.
For cast bullet loadings against pests and noxious animals of many varieties, either a 200 grain WC or SWCHP at 850-900 fps will work just dandy. As Charter Arms used to advertise, "Carries like a .38, Hits like a .45". GT Bullets sells an excellent 190 grain SWCHP that in my testing at 870 fps, expands to over .630 diameter and penetrates 13"+ in ballistic media. The WC will penetrate to 20"+ - no expansion but a lot of disruption. For these loads I have used HS-6 and Power Pistol, but currently prefer Accurate No.5.
Like all things .44 Special, the powder companies and loading manuals will show weak loads that are well below even the anemic original 1908 ballistics, which have never been updated. Brian Pearce authored an excellent article in Handloader concerning the .44 Special using modern propellants and discussing modern pressure levels. You can find loads in that article, and also on the loadata.com site if you are a member.
Jacketed bullets are a different story but you asked specifically about cast. Be safe and enjoy loading and shooting for the Bulldog.