Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
KaneTW
Dec 2, 2011

DARPA Dad posted:

how in the hell are unsigned ints and unsigned longs incompatible types according to MSVC? i had to cast the former to the latter just now. i have never come across something this dumb.

edit: ok i have come across lots of dumber poo poo but this was just baffling in the sense that i was convinced for a good 30 minutes that my compiler was just lying to me

edit2: it wasn't even a warning. it threw a full out error

Are you using a current MSVC? That shouldn't happen.

Adbot
ADBOT LOVES YOU

DARPA Dad
Dec 9, 2008

KaneTW posted:

Are you using a current MSVC? That shouldn't happen.

VS Ultimate 2013. should i be using the 2015 beta?

DARPA Dad fucked around with this message at 05:16 on Feb 9, 2015

PRADA SLUT
Mar 14, 2006

Inexperienced,
heartless,
but even so
I'm not sure if I should put this in C or in Engineering or in Arduino, since it's all three.

I need help with an Arduino project. My Arduino has an ultrasonic distance sensor hooked up to a servo which spins it in a 180 arc and writes data to an int array at various points detailing the distance in centimeters (essentially mapping out a room using a spinning turret).

So, it would be like array[] with values like 26,26,24,22,28,30,33,35, etc (~40 in total), which is the distance between the direction the sensor is facing and the room it's mapping.

My problem comes in that sometimes theres a glitch and a single measurement is completely off. So, my values might read like 26,26,24,600,28,30,6,35.

In pseudo-code, this is how it takes measurements:

C++ code:
for (int i = 0; i < 50; i++) {
	turnSensor[i];
	(get distance from sensor);
	arr[i] = distance;
}
Sample array with "bad" values highlighted:
arr[n]
[0] 26
[1] 26
[2] 24
[3] 600
[4] 28
[5] 30
[6] 6
[7] 35

What I'm wondering is an idea of some way through code to isolate those bad measurements and have my Arduino then go back to those points and re-take the data. I was thinking that maybe some way to have it look at the positions of the numbers before and after it, and then if it's off within 10% or something, re-take that measurement? The problem then is that it would re-take numbers before the bad data, since the average of the bad data and the good data before it would be 10% different. Maybe a way to note that if the previous data check was good, but the current data check was bad, then i++, like it needs 2 bad data checks in a row? Because errors increase at distance, I can't just use a straight +/- 2 or 3 or whatever, since at a longer distance, a 2-3 cm difference is acceptable, when you're measuring an entire room, and a 600 might be a legit measurement if it's looking at something far away, so I can't just exclude and re-take all high measurements, I need some comparative way to figure it out. It's extremely unlikely for two bad measurements to happen in a row, so I'm hoping to have some way to differentiate a glitch with a legit measurement (like there's something in front of the Arduino). Maybe only correct if it's a single outlier, not multiple in a row that are similar to each other?

I could have it take data multiple times and then discard data that's way off the average, but that takes twice as long as just (it's about 8 seconds a scan now) trying to isolate the problem points and re-taking those.

I guess my problem is that I don't understand how I would even go about this, let alone code it.

PRADA SLUT fucked around with this message at 05:57 on Feb 9, 2015

nielsm
Jun 1, 2009



Make a low-pass filter over the data set to get a smoothed curve, then compare each original data point to the smoothed curve and flag large deviations.

(A simple low-pass filter is just a moving average.)

sarehu
Apr 20, 2007

(call/cc call/cc)
Is spinning around twice really going to be slower than spinning to one or two points and taking specific measurements? Does taking measurements really take appreciable time, compared to the rotation? If the rotation is what's slow, you're going to 1.5x your time if there's one bad point and 2x it if there's two bad points (generally) anyway. (What's the average number of bad points per run anyway?). Why can't you just take multiple measurements at a point, in a single rotation, if rotation is the slow part? Also, why can't you catch weird measurements on the fly, while the measuring device is still pointing close to in that direction?

You can recognize a weird measurement at point n if x_n - x_{n-1} is way bigger than x_{n-1} - x_{n-2} and x_{n+1} - x_{n-1} is not vastly different from 2*(x_{n-1} - x_{n-2}). Note the undesirable asymmetry of what I just said and you could probably improve it.

You can handle the error margin depending on distance by putting the values through an appropriate function before looking for anomalies. For example, if the relative difference between values is generally proportional to their size (which is what would make sense for a perfectly accurate sensor, anyway), you could map the values x to log(x) before analyzing them for anomalies. Or you could just magnify your bar of an anomalous difference to be proportional to the values in question, or you could compare ratios of values instead of differences. (These are all the same thing in principle and two are the same in outcome.)

Edit: note that a low pass filter will result in flagging jump discontinuities. You want to flag point discontinuities. There is probably a really simple filter for that too though, like y_n = 2*x_n - x_{n-1} - x_{n+1}, though I think you'd want to throw the x_n's through a log function first.

sarehu fucked around with this message at 06:38 on Feb 9, 2015

PRADA SLUT
Mar 14, 2006

Inexperienced,
heartless,
but even so
I'd say the error rate is roughly 5%. This is problematic though, because eventually I want it to track a moving object, so I need an accurate "base" measurement for it to compare against. It's essentially going to take a baseline measurement and then track the room, looking for new discrepancies (by comparing the two sets of data).

In order for it to get 5% errors, I need ~10 seconds per sweep. I can sweep faster, but the errors will increase far greater (I get around 80% error at 2 second sweeps).

sarehu posted:

Is spinning around twice really going to be slower than spinning to one or two points and taking specific measurements? Does taking measurements really take appreciable time, compared to the rotation? If the rotation is what's slow, you're going to 1.5x your time if there's one bad point and 2x it if there's two bad points (generally) anyway. (What's the average number of bad points per run anyway?). Why can't you just take multiple measurements at a point, in a single rotation, if rotation is the slow part? Also, why can't you catch weird measurements on the fly, while the measuring device is still pointing close to in that direction?

You can recognize a weird measurement at point n if x_n - x_{n-1} is way bigger than x_{n-1} - x_{n-2} and x_{n+1} - x_{n-1} is not vastly different from 2*(x_{n-1} - x_{n-2}). Note the undesirable asymmetry of what I just said and you could probably improve it.

You can handle the error margin depending on distance by putting the values through an appropriate function before looking for anomalies. For example, if the relative difference between values is generally proportional to their size (which is what would make sense for a perfectly accurate sensor, anyway), you could map the values x to log(x) before analyzing them for anomalies. Or you could just magnify your bar of an anomalous difference to be proportional to the values in question, or you could compare ratios of values instead of differences. (These are all the same thing in principle and two are the same in outcome.)

I think the problem is that in order for me to tell if it's a weird measurement, I would need to know the value of the next measurement. For example, if I set it on the floor of an empty room but put a coffee cup next to it on one side, it will throw a weird-looking measurement at me when it encounters the cup, but it won't know if that's a legit measurement until it scans more (and sees that there are multiple in a row, since errors really never happen right next to each other, at the same level).

I could take multiple measurements, but I was hoping that there would be a way that was quicker. The slowest part is the mechanical part--waiting for the servo to physically spin the sensor around, and then the sonar pulse to go out and come back. That's why I was hoping to fix it in software as much as I could, and only re-scan the bad sections. That way I can just spin it around to 2 or 3 specific points and re-take, rather than spin it to 50 points to take an entirely new dataset again.

mmkay
Oct 21, 2010

Subjunctive posted:

Still better than strncpy, which nobody should ever use.

Why is it so bad? Is it the fact that it won't null terminate a string over the specified size the problem, or what?

Jabor
Jul 16, 2010

#1 Loser at SpaceChem

mmkay posted:

Why is it so bad? Is it the fact that it won't null terminate a string over the specified size the problem, or what?

The whole point of not using strcpy is so that you don't introduce a security vulnerability if someone gives you a string too large for your buffer. strncpy does absolutely dick to solve that - if someone gives you a string too large for your buffer, you still end up in a compromised, exploitable state.

Sure, you can do some extra effort to make strncpy safe after the fact, but the better plan is to just use a function that does it right in the first place instead of something that gives you a false sense of security.

Jewel
May 2, 2009

Jabor posted:

The whole point of not using strcpy is so that you don't introduce a security vulnerability if someone gives you a string too large for your buffer. strncpy does absolutely dick to solve that - if someone gives you a string too large for your buffer, you still end up in a compromised, exploitable state.

Sure, you can do some extra effort to make strncpy safe after the fact, but the better plan is to just use a function that does it right in the first place instead of something that gives you a false sense of security.

How does this work, exactly? Not saying you're wrong, just wondering. You have a buffer 100 big, you copy a 200 length string into it with strncopy(dest, source, 100) and cap it off with a null char at the end, what's the issue here, exactly?

TZer0
Jun 22, 2013

Jabor posted:

The whole point of not using strcpy is so that you don't introduce a security vulnerability if someone gives you a string too large for your buffer. strncpy does absolutely dick to solve that - if someone gives you a string too large for your buffer, you still end up in a compromised, exploitable state.

Sure, you can do some extra effort to make strncpy safe after the fact, but the better plan is to just use a function that does it right in the first place instead of something that gives you a false sense of security.

If someone gives you a string too large for your buffer, then it has nothing to do with strncpy in the first place in my opinion. Unless you do things like strncpy(dest, src, strlen(src)) (equivalent to strcpy(dest, src)) I'd say that the problem lies in how the data is being read rather than the use of strncpy.


Edit:
I'm not saying that you *should* use strncpy over the other alternatives.

Jabor
Jul 16, 2010

#1 Loser at SpaceChem

Jewel posted:

How does this work, exactly? Not saying you're wrong, just wondering. You have a buffer 100 big, you copy a 200 length string into it with strncopy(dest, source, 100) and cap it off with a null char at the end, what's the issue here, exactly?

The fact that you have to manually cap it off with a null character yourself isn't a good enough reason to say "you should use something that actually works as intended"?


TZer0 posted:

If someone gives you a string too large for your buffer, then it has nothing to do with strncpy in the first place in my opinion. Unless you do things like strncpy(dest, src, strlen(src)) (equivalent to strcpy(dest, src)) I'd say that the problem lies in how the data is being read rather than the use of strncpy.

The problem is if you do strncpy(dest, src, sizeof(dest)) and leave it at that. Awesome, problem solved, you didn't use strcpy, totally immune to buffer overflows! Except, of course, that you're still totally hosed if someone does give you a string longer than the buffer.

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

mmkay posted:

Why is it so bad? Is it the fact that it won't null terminate a string over the specified size the problem, or what?

Yes, it doesn't always produce a legal string.

limip
Oct 24, 2003

DARPA Dad posted:

how in the hell are unsigned ints and unsigned longs incompatible types according to MSVC? i had to cast the former to the latter just now. i have never come across something this dumb.

edit: ok i have come across lots of dumber poo poo but this was just baffling in the sense that i was convinced for a good 30 minutes that my compiler was just lying to me

edit2: it wasn't even a warning. it threw a full out error

Was this dealing with pointers to unsigned int and unsigned long? I think the only times I've seen MSVC messages that reference "incompatible types" is when dealing with pointers.

ExcessBLarg!
Sep 1, 2001
Drepper's argument against strlcpy is that you should know the size of the source string anyways to avoid truncation errors, in which case you can just use memcpy to do the actual copy. That said, you can detect truncation errors with strlcpy anyways, and there's times when you want to copy a string and you genuinely don't care about truncation errors. Honestly, I think he's wrong, if only because by introducing strlcpy into GNU libc, it would strictly discourage the use of strcpy or strncpy and a number of other inherently unsafe or otherwise frequently misused approaches.

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

Drepper is very very smart, but not always very wise.

DARPA Dad
Dec 9, 2008

limip posted:

Was this dealing with pointers to unsigned int and unsigned long? I think the only times I've seen MSVC messages that reference "incompatible types" is when dealing with pointers.

I was assigning a member of a struct (using the . operator) to the return value of a function that took a reference

DARPA Dad fucked around with this message at 19:33 on Feb 9, 2015

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

DARPA Dad posted:

I was assigning a member of a struct to the return value of a function that took a reference

That returned a reference? Sounds like pointer use, effectively.

DARPA Dad
Dec 9, 2008

Subjunctive posted:

That returned a reference? Sounds like pointer use, effectively.

I'll have to check the code again but my gut feeling is I wasn't returning a reference, only passing one in as one of two parameters

Private Speech
Mar 30, 2011

I HAVE EVEN MORE WORTHLESS BEANIE BABIES IN MY COLLECTION THAN I HAVE WORTHLESS POSTS IN THE BEANIE BABY THREAD YET I STILL HAVE THE TEMERITY TO CRITICIZE OTHERS' COLLECTIONS

IF YOU SEE ME TALKING ABOUT BEANIE BABIES, PLEASE TELL ME TO

EAT. SHIT.


I know there's been some recent discussion about this, but what would be a good c++ IDE for linux? I'd just use VS but I only have a few weeks and I'd rather not deal with any portability issues (it's a compiler so there's bound to be some). My first instinct is to use eclipse since I like using it for java (netbeans might be better but whatever) and python. But I've heard it's really bad? I don't want to use sublime/vim/emacs because I want an actual IDE, not a glorified text editor.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe

ExcessBLarg! posted:

Drepper's argument against strlcpy is that you should know the size of the source string anyways to avoid truncation errors, in which case you can just use memcpy to do the actual copy. That said, you can detect truncation errors with strlcpy anyways, and there's times when you want to copy a string and you genuinely don't care about truncation errors.

There are very few times this is acceptable, especially if you have properly localized software that uses UTF-8. So many bugs in other teams have been assuming that byte-based truncation is safe and cool to do.

Subjunctive
Sep 12, 2006

✨sparkle and shine✨

Don't use str* on UTF-8, I guess.

The_Franz
Aug 8, 2003

Private Speech posted:

I know there's been some recent discussion about this, but what would be a good c++ IDE for linux? I'd just use VS but I only have a few weeks and I'd rather not deal with any portability issues (it's a compiler so there's bound to be some). My first instinct is to use eclipse since I like using it for java (netbeans might be better but whatever) and python. But I've heard it's really bad? I don't want to use sublime/vim/emacs because I want an actual IDE, not a glorified text editor.

If you know Eclipse and are comfortable with it, use it.

Otherwise there is QtCreator, which is probably the least-bad cross-platform IDE right now as it's relatively fast and everything works. There is also CLion from JetBrains that is shaping up to be really nice, but it still in beta and may not fit your needs at the moment (CMake projects and gdb debugging only).

Edison was a dick
Apr 3, 2010

direct current :roboluv: only

Private Speech posted:

I know there's been some recent discussion about this, but what would be a good c++ IDE for linux? I'd just use VS but I only have a few weeks and I'd rather not deal with any portability issues (it's a compiler so there's bound to be some). My first instinct is to use eclipse since I like using it for java (netbeans might be better but whatever) and python. But I've heard it's really bad? I don't want to use sublime/vim/emacs because I want an actual IDE, not a glorified text editor.

I've heard people recommend KDEvelop. I wouldn't know, I'm mostly here for C chat, not C++ chat.

hooah
Feb 6, 2006
WTF?
My Viterbi tagging algorithm is running now (and I fixed a problem where it assigned the same tag to every word except the first and last of the sentence), but I don't think it's doing as well as it should. As an example, when it sees this sentence

quote:

The Arizona UNKA Commission authorized an 11.5 % rate increase at Tucson Electric Power Co. , substantially lower than recommended last month by a commission hearing officer and barely half the rise sought by the utility .
the output is:

quote:

the VBP arizona NNP unka SYM commission NN authorized JJ an NNP 11.5 CD % JJ rate VB increase VBP at NNP tucson NNP electric NN power SYM co. NNP , SYM substantially RB lower NNP than RBR recommended VBD last NN month NNP by RB a IN commission FW hearing NN officer NNP and IN barely RB half RB the NN rise VBP sought VBD by RB the NN utility NNP . SYM

Whereas the ground truth is

quote:

The DT Arizona NNP Corporations NNP Commission NNP authorized VBD an DT 11.5 CD % NN rate NN increase NN at IN Tucson NNP Electric NNP Power NNP Co. NNP , , substantially RB lower JJR than IN recommended VBN last JJ month NN by IN a DT commission NN hearing NN officer NN and CC barely RB half PDT the DT rise NN sought VBN by IN the DT utility NN . .

My code is here. If anyone were so kind as to look it over, is there something I'm just overlooking? Unfortunately the professor is not helping us with implementation details.

Diametunim
Oct 26, 2010
Could anyone explain to me why I only remove one value per call when I call my intersect function? (Lines 313-325) and why my subtract function causes a seg fault? (Lines 333-342) My remove function works just fine when I call it on it's own, if I call it from within my subtract function I get a seg fault though. Here's the code in a Gist.

https://gist.github.com/anonymous/8aeca6e1b8b036c5bbe0

nielsm
Jun 1, 2009



Why does your contains function return an int instead of a bool? It makes no sense, fix that just to make the rest of the code more readable.

When you loop over an array from 0 to n-1 and then remove elements from the array at the same time, the length of the array changes. In fact, if you remove element i from the array, by shifting all following elements down one, then your loop variable i now refers to the element originally following the removed one.
I'm not sure about the segfault, but the reason you only see one element removed by intersect is probably due to your test data causing the incorrect behavior (skipping elements) to manifest as only a single element removed.


Oh and by the way, line 214, "dummy value"? The behavior there is correct but it's definitely not "dummy", explain why it's the correct thing to do.

Fergus Mac Roich
Nov 5, 2008

Soiled Meat

Diametunim posted:

Could anyone explain to me why I only remove one value per call when I call my intersect function? (Lines 313-325) and why my subtract function causes a seg fault? (Lines 333-342) My remove function works just fine when I call it on it's own, if I call it from within my subtract function I get a seg fault though. Here's the code in a Gist.

https://gist.github.com/anonymous/8aeca6e1b8b036c5bbe0

I'm just dipping my toes into C++ from C89, which has different rules for where you can declare variables, but this looks suspicious to me in the remove function:

code:
for(int i = 0; i < capacity; i++)
   {
      if(data[i] == anInt)
      {
         for(int i; i < used - 1; i++)
I'm not 100% clear on scoping rules for nested for loops in C++, so I'll assume it's legal to re-use the name, but at the very least shouldn't you provide an initial value for i in the second loop?

fritz
Jul 26, 2003

The_Franz posted:

There is also CLion from JetBrains that is shaping up to be really nice, but it still in beta and may not fit your needs at the moment (CMake projects and gdb debugging only).


I'm liking clion a lot, I've only run into a couple bugs that are at the level of 'annoying', and it's got me using cmake which is pretty great.

nielsm
Jun 1, 2009



Fergus Mac Roich posted:

I'm just dipping my toes into C++ from C89, which has different rules for where you can declare variables, but this looks suspicious to me in the remove function:

code:
for(int i = 0; i < capacity; i++)
   {
      if(data[i] == anInt)
      {
         for(int i; i < used - 1; i++)
I'm not 100% clear on scoping rules for nested for loops in C++, so I'll assume it's legal to re-use the name, but at the very least shouldn't you provide an initial value for i in the second loop?

I missed that one. True, the inner loop has a separate i variable shadowing the outer one, and it happens to be undefined at every iteration of the inner loop.

You ought to be compiling your code with warnings enabled. Your compiler will warn you about variables shadowing others, as well as use before definition.

(Variables declared in a control statemens like for, if, while are scoped to that statement and the controlled block.)

ullerrm
Dec 31, 2012

Oh, the network slogan is true -- "watch FOX and be damned for all eternity!"

Subjunctive posted:

Don't use str* on UTF-8, I guess.

Eh, that's probably an exaggeration. The entire point of UTF-8 is that it's meant to be compatible with C strings. Sure, if you truncate it in the middle of a multibyte sequence, you'll end up with that codepoint being undecodable... but truncating ANY string needlessly is pretty bad, whether it's UTF-8 or not. std::string exists for a reason.

Really, there's just one thing you need to worry about with UTF-8 in particular, assuming you aren't screwing up string handling elsewhere: strlen() will be measuring the number of bytes, and not the number of Unicode codepoints. All other C string functions should work as intended.

The rest of the concerns with UTF-8 are really concerns about Unicode in general, namely:

* There's no strong correlation between the count of Unicode codepoints in a string, and the count of visual characters (aka graphemes) in a string. Multiple Unicode characters can combine into a single grapheme, and some Unicode codepoints encode more than one grapheme. Some Unicode characters don't even encode visual information -- for example, U+FFF9 is used as a prefix to denote context / metadata that could be displayed as a footnote or a tooltip or whatever.

* There can be more than one way to encode a grapheme. A lowercase o with an umlaut could be written as U+00F6 ('LATIN SMALL LETTER O WITH DIAERESIS'), or as U+006F ('LATIN SMALL LETTER O') followed by U+0308 ('COMBINING DIAERESIS'). When you're implementing anything that compares strings -- sorting, text search, etc -- you should probably convert all strings involved to a canonical format. See also: http://en.wikipedia.org/wiki/IDN_homograph_attack

Fergus Mac Roich
Nov 5, 2008

Soiled Meat
-Wall -pedantic for life.

fritz
Jul 26, 2003

nielsm posted:


You ought to be compiling your code with warnings enabled. Your compiler will warn you about variables shadowing others, as well as use before definition.


I don't think -Wshadow is in gcc's standard -Wall block, maybe not even in -Wevery, but I'm not in a situation to test that right now.

Zopotantor
Feb 24, 2013

...und ist er drin dann lassen wir ihn niemals wieder raus...

Fergus Mac Roich posted:

-Wall -pedantic for life.

Look at this poser who doesn't use -Werror. :colbert:

Fergus Mac Roich
Nov 5, 2008

Soiled Meat

Zopotantor posted:

Look at this poser who doesn't use -Werror. :colbert:

Think I'm going to start using this and -Wextra. C and its compilers sort of assume the programmer is always right, whereas I am never right and need them to be as paranoid as possible.

b0lt
Apr 29, 2005

Fergus Mac Roich posted:

Think I'm going to start using this and -Wextra. C and its compilers sort of assume the programmer is always right, whereas I am never right and need them to be as paranoid as possible.

There's -Weverything too!

Blotto Skorzany
Nov 7, 2008

He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip
I don't recall which, but there's at least one compiler that has both -Wall and -Wactually-all

Meat Beat Agent
Aug 5, 2007

felonious assault with a sproinging boner
You might be thinking of PHP's E_ALL and E_ACTUALLY_ALL (though as that page points out, "E_ACTUALLY_ALL" doesn't actually exist)

Edison was a dick
Apr 3, 2010

direct current :roboluv: only

Zopotantor posted:

Look at this poser who doesn't use -Werror. :colbert:

I will fight anyone who forces that in their Makefiles. It makes it a pain in the arse when you want to rebuild your whole distro with a new compiler, since it always finds new warnings.

Qwertycoatl
Dec 31, 2008

Edison was a dick posted:

I will fight anyone who forces that in their Makefiles. It makes it a pain in the arse when you want to rebuild your whole distro with a new compiler, since it always finds new warnings.

It should be easy to temporarily turn off with a local change, but I've found when it's not there, people introduce new warnings, decide they're harmless and it's not worth the effort of fixing them right now (maybe later, in the mythical time when things are quieter), until there's so much spam you don't notice the nasty warnings any more.

Adbot
ADBOT LOVES YOU

Presto
Nov 22, 2002

Keep calm and Harry on.
I would not have been able to build with -Werror at my last job due to warnings coming from system and boost headers.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply