I'm interested in learning how to scale my web app beyond a single server. Ideally I'd like it to be hosted in multiple regions with GeoDNS taking you to the closest one. From what I can tell, the database replication portion of this is probably gonna be the most difficult part. It's tempting to use Amazon RDS, and I may end up going down that route, but I'd still like to learn how to do it myself. Is it feasible to have MySQL databases replicated across different regions? Do writes always go to the master? How bad does it get when something goes wrong? What's a good resource for finding answers to questions like these?
|
|
# ¿ Jan 16, 2014 00:41 |
|
|
# ¿ May 14, 2024 14:53 |
chmods please posted:1) Once the database is built with SQLite why can't you query it locally instead of having to go out over the Internet? I'd say pick SQLite, no need to mess with MySQL right now. 1 million rows is nothing, if you manage to get a slow query, then figure out how to optimize that specific one. Don't add indexes to everything, you'll be worse off. fletcher fucked around with this message at 09:16 on Feb 25, 2014 |
|
# ¿ Feb 25, 2014 08:15 |
dammitcharlie posted:Sorry, maybe I don't understand quite how this all works, but I can't do it locally because I don't want the program to be so massive; I suppose it could be but my professor said that she'd prefer we keep the programs relatively small. Your data shouldn't be stored with your code. Here are a couple options I would explore rather than trying to go client/server with MySQL: * Can the data be generated by a small script so you can populate your database when needed? * If it's some existing dataset and you can't generate it on the fly, can you just store a CSV of it on a thumbdrive/dropbox/ftp/whatever, and import into SQLite as needed?
|
|
# ¿ Feb 25, 2014 20:13 |
dammitcharlie posted:The data will be generated by scraping websites over the next few days. So I guess I should scrape the data into a CSV file and then store that CSV with the program which will load it into sqlite upon starting? Yup I'd go with something just like that. Bonus points if you can simply specify a URL for the data source and if the data hasn't been downloaded before, it downloads it for you. That way the same steps for running it the first time can be used for running it a second time. If you have time you might also want to explore having your scraper automatically compress the csv into a zip file, and then your program can uncompress it after it downloads it. I haven't used it before but it looks like Python has a nice interface for working with zip files.
|
|
# ¿ Feb 25, 2014 22:06 |
ManoliIsFat posted:Ya, in SQL, you test for null with "[column] IS NULL" Gets me every time. Why is it like that?
|
|
# ¿ Mar 10, 2014 22:19 |
Kumbamontu posted:NULL isn't a number, it's the inherent lack of knowledge of a value. Therefore, the expression 'NULL =' anything really doesn't make sense, as you don't know if it's 0, a billion, -3.14, undefined, etc. But in programming languages like Java you can do null comparisons with ==
|
|
# ¿ Mar 10, 2014 23:05 |
Agrikk posted:I have what I think amounts to a data warehousing question for SQL Server 2012 and I'm not sure how to approach it. Does it have to generate it in real time on demand? Maybe just generate the summary report periodically with a cron job and serve up the generated report.
|
|
# ¿ Apr 29, 2014 22:11 |
Something like this maybe?code:
|
|
# ¿ May 7, 2014 20:15 |
Trying to do everything with database queries seems like it will be pretty cumbersome. I would just write a little python program that uses a sqlite database or something to massage your data.
|
|
# ¿ May 8, 2014 21:53 |
the posted:Is this a good place to ask stupid questions about SQL queries to a Salesforce database? Yea I think this would be an appropriate thread for that. I don't know nuthin bout no SQL in Salesforce though. Now, SOQL on the other hand...
|
|
# ¿ Jun 12, 2014 19:56 |
the posted:Haha yes, thanks. I'm using Beatbox in Python to access the database. Yup that's doable. Make sure you turn on History Tracking for the Account object if you haven't already. Setup->Customize->Accounts->Fields then click Set History Tracking button and select which fields you want tracked. Now you can execute a query like: code:
|
|
# ¿ Jun 12, 2014 20:20 |
the posted:edit: I'm reading through the SOQL help file, and what I'm really looking for is a listing of what all the field types are referring to. Like, how was I supposed to know that the Billing City is referred to as "billingCity" without seeing it in an example? For that you have to use describe calls: https://www.salesforce.com/us/developer/docs/api/Content/sforce_api_calls_describesobjects.htm It will return everything about that object: https://www.salesforce.com/us/developer/docs/api/Content/sforce_api_calls_describesobjects_describesobjectresult.htm#topic-title In the array of fields that it returns you can see the label & name (Billing City vs. billingCity): https://www.salesforce.com/us/developer/docs/api/Content/sforce_api_calls_describesobjects_describesobjectresult.htm#i1427375 edit: or if you just want to look at it through the Salesforce UI go to Setup->Customize->Accounts->Fields and it will list all the Field Labels & Field Names
|
|
# ¿ Jun 12, 2014 20:22 |
the posted:Can I combine two selections? Like for that example, say I wanted to select the account history, but only for accounts located in New York. How would I do this? My inclination is to do something like: Yup, you can do that: code:
code:
|
|
# ¿ Jun 12, 2014 20:57 |
the posted:fletcher, I'm hoping you can be my savior again. I'm not sure how good you are with Python, but you indicated that you use beatbox so I'm hoping you're the perfect person to help me. I've never used beatbox before, the bulk of my experience is using the Salesforce SOAP API via Java code. It looks like you're actually using beatboxxx rather than beatbox? I couldn't get the test code to run with the latter. What do you plan on doing with these records? Typically you don't want to load them all into memory at once, you process them in chunks and then use queryMore to get the next chunk from Salesforce. This keeps your memory footprint low. At any rate, if you truly do need an all_records: Python code:
fletcher fucked around with this message at 18:30 on Jun 17, 2014 |
|
# ¿ Jun 17, 2014 18:27 |
the posted:I want to do data analysis with them. Not sure if there is a simple way to do that. You may have to go further down the route you are currently on and query/process the data manually yourself. Just be sure to leverage the built in relationships as much as you can to avoid extra queries. And to be honest, I only have experience with Salesforce from the developer perspective. When it comes to actually using it like an end user things get a little fuzzy, I've never done anything with those Lead & Opportunity objects. Could you do something with the delta between the CreatedDate & ConvertedDate? code:
|
|
# ¿ Jun 17, 2014 21:36 |
the posted:I've got a LeadSource stored as "I'm an existing client", but when I try to query it it says that it's an invalid query. Is it stored somehow without the apostrophe? I think you need some more backslashes. You want the query to end up being: code:
code:
|
|
# ¿ Jun 19, 2014 22:40 |
PhantomOfTheCopier posted:Wait, you're using Python and passing a literal string into a database? Have you heard of SQL injection? There must be seventy thousand modules for data handling in Python by now. It's Salesforce, you pretty much have to use string literals. At any rate, SOQL is read only. You can't CREATE/UPDATE/DELETE, those use a separate API. And there's no risk of querying for data you don't have access to, that is all handled by the Salesforce user security model.
|
|
# ¿ Jun 20, 2014 00:12 |
the posted:How do I format something like this: Kumbamontu's response is the better solution, but I just wanted to point out that you weren't too far off. Just need to add some parentheses: code:
|
|
# ¿ Jul 1, 2014 21:36 |
the posted:You can view my history, but I'm doing queries in Salesforce using Python. The SQL queries are basically no different, but they're embedded in Python code. Use a subquery (assuming you can perfrom 'some stuff' where additional SOQL) code:
Python code:
the posted:fletcher would know how to do this. Maybe flip it around and query for Opportunity fields through OpportunityContactRole: Python code:
|
|
# ¿ Jul 8, 2014 16:23 |
Essential posted:How important is it to have a CreatedDate & ModifiedDate columns on tables? I've got a guy here telling me that EVERY table, without exception should have those 2 columns. We're adding a couple tables and right now I don't care about CreatedDate/ModifiedDate on a few of the tables and I can't ever see it being needed on those tables. His argument is that what IF at some point in the future I want it and even if I don't, it isn't going to affect anything negatively. Personally, I like having audit fields on all of my tables (created date, created by, modified date, modified by), even if I don't foresee using that information (and there's been a few times where I'm glad I had it).
|
|
# ¿ Jul 16, 2014 19:05 |
the posted:In Salesforce, I have an Opportunity entity that has many different custom fields associated with it. What I'm finding out is that many of the custom fields are throwing errors when I make SOQL requests to them. But some aren't. And I can't figure out what is the difference, since they're all custom fields. Sounds like a field level security issue. Go to Setup->Manage Users->Users. Find the user you are executing the queries as and click on the name of the Profile. Then scroll down to the "Field-Level Security" section.
|
|
# ¿ Jul 22, 2014 20:06 |
the posted:Yep, that ended up being it! Turns out that literally the only group not given access to that field, out of the two dozen or so groups, was mine I've been there my friend! In general, how is your Salesforce stuff going? Making progress on whatever it is you are building?
|
|
# ¿ Jul 22, 2014 20:16 |
the posted:Yeah, the main issue I've been running into is twofold: We also ran into limitations of what we could do with Salesforce reports and ended up going down the route of Java + SOAP API + MySQL to accomplish what we needed to do. Salesforce reports are nice but sometimes you need to break out of them for more flexibility. the posted:fletcher, while you're here, I'm trying to query a list of Opportunities and find out whether or not they have a Primary Contact. I don't see that there's a specific field for that. It looks like I can query OpportunityContactRole to see if a contact is a Primary Contact, but the problem there is that there are some Opportunities that don't have Contacts at all, so I'd have to grab a list of all Opportunities, then a list of all OpportunityContactRoles, and then filter out what doesn't match. Sounds like you might be able to use a subquery to accomplish this? Something like: code:
|
|
# ¿ Jul 22, 2014 22:08 |
I think that query handles A, B, and C though. For case A, the subquery would simply have a row count of 0.
|
|
# ¿ Jul 23, 2014 01:51 |
Malcolm XML posted:Left join that poo poo son This is Salesforce, there is no JOIN keyword. The subquery version that I posted should be the equivalent of a LEFT OUTER JOIN
|
|
# ¿ Jul 23, 2014 18:49 |
the posted:fletcher, do you know why an admin would get an "Insufficient Privileges" message when trying to access any object page in Salesforce? I would assume they'd have access to everything. Are you referring to the standard list view for an object that uses the 3 letter object key prefix? (i.e. https://zzz.salesforce.com/001 for Accounts) That list view is not supported for many objects, I think it might be the "layoutable" property that determines that, can't remember for sure though:
|
|
# ¿ Jul 25, 2014 19:22 |
EAT THE EGGS RICOLA posted:I have a postgres database that I need to replicate across a dozen servers. Use a VPN or tunnel over SSH. Definitely don't open up the database port to the world!
|
|
# ¿ Aug 27, 2014 18:53 |
Why am I not getting a permission denied error here? I already did a FLUSH PRIVILEGES from the root mysql account before trying this:code:
|
|
# ¿ Nov 26, 2014 22:06 |
Anaxandrides posted:FILE permissions are for files on the server locally -- remote file access is based on rights of the server proxy user to the remote machine. This was with mysqld running on my localhost and /tmp/test_data.csv is also on my local machine, shouldn't the FILE permission cover that?
|
|
# ¿ Nov 26, 2014 23:57 |
|
|
# ¿ May 14, 2024 14:53 |
Anaxandrides posted:Why did you use double backslashes, then? I can't say on that one -- is there a bug on MySQL that causes it to see this as a remote server? Hmmm can't remember why I had the double backslashes, it seems to work with just singles though. Still can't get it to throw a permissions error though.
|
|
# ¿ Dec 1, 2014 23:15 |