Question C++/Template specialization

[MeDiCS]

Summary of what I've learned in this thread:

• A template class/method/function should be implemented in a header file to be included in every cpp file where it's needed. It can't be implemented in a separate cpp file (as you'd normally do for normal functions and methods), as the compiler need to know for which types it need to generate code for.
• NEVER ask questions if you're in need of a good night's sleep :thumbup:.

Original Post:

I'm having problems with template specializations:

class Test
{
    template<class T> T get_value ();
};

template <class T> T Test::get_value ()
{
    //Some code
}

template <> string Test::get_value ()
{
    //Specialized code
}

The problem is that only the specialization get compiled. When I try to use the general case, I get 'error LNK2001: unresolved external symbol'.

Test a;
a.get_value<string> (); //Works
a.get_value<double> (); //Doesn't work

I've also opened the generated object file, and the only symbol declared is the specialized function, but not the general case.

Using VC2008 BTW.

 

[Urwumpe]

I think you have to use

template <class T> T Test::get_value()
{

}

But I am not 100% sure if this is really the cause. The order of the functions could also be important, but I am not sure if it was "general case first" or "Special case first", I don't work on templates without a reference manual, since I rarely need them.

 

[MeDiCS]

I think you have to use

template <class T> T Test::get_value()
{

}

True. It's actually a typo.

---------- Post added at 12:43 PM ---------- Previous post was at 12:41 PM ----------

The order doesn't seem to change anything. The problem is actually in the template definition itself, and not in the specialization as I've removed the specialized function while still getting the same error. I'll test with MinGW.

 

[Urwumpe]

I think you also have to write

template<> 
string get_value<string>()
{
}

But I am not sure with that.

 

[martins]

I don't know how VC handles template instantiations, but in gcc I usually don't rely on the compiler instantiating template types automatically. Instead, I instantiate all required types manually, by adding something like this to the cc file containing the class definition:

template class Test<double>;
template class Test<float>;
template class Test<int>;
// etc.

(Note that specialisations shouldn't be instantiated in this way, since the compiler instantiates them directly, just like normal member functions).

In any case, if you want automatic instantiation, doesn't the entire template class definition have to be in the header file, so that the compiler has it available when needed?

 

[cjp]

In any case, if you want automatic instantiation, doesn't the entire template class definition have to be in the header file, so that the compiler has it available when needed?

I also think it has to be. In fact, AFAIK this is the only way how the 'general case' function is used. I certainly would not expect it to be present in an object file: only (manually or automatically generated from the template) specialized versions go there.

I rarely use manually created specialized cases, and I usually let the compiler generate specialized cases automatically from the template definition when it needs them. The only issue is that this probably leads to a lot of code duplication in the executable, if the same specialized function is added to different object files.

But templates are one of the most advanced topics in C++, and while I have some basic experience with them, there are probably still some things I don't know.

 

[Enjo]

I'm not sure if I understand correctly.
You have a header file in which you only declare templates and then you specialise them, right?

From my experience, you can't declare templates without defining them. You must either put a template's definition (without specialisation!) inside a header file (increases compile time), or if you're making a closed source library, you can simply put such a template in a cpp file. I imagine that to use it in other files, you'd have to declare the template somewhere, using the external keyword. Probably in the headers, but I'm not sure if you can even declare a template with the external keyword.

Tell me how you get on.

 

[MeDiCS]

I think you also have to write

template<> 
string get_value<string>()
{
}

But I am not sure with that.

Nope. That's just a formatting style. The extra '<string>' is also not needed when the compiler can infer the type of the template.

I don't know how VC handles template instantiations, but in gcc I usually don't rely on the compiler instantiating template types automatically. Instead, I instantiate all required types manually, by adding something like this to the cc file containing the class definition:

template class Test<double>;
template class Test<float>;
template class Test<int>;
// etc.

(Note that specialisations shouldn't be instantiated in this way, since the compiler instantiates them directly, just like normal member functions).

In any case, if you want automatic instantiation, doesn't the entire template class definition have to be in the header file, so that the compiler has it available when needed?

Erm, well, that's a template class. My problem is regular classes with a template method.

I also think it has to be. In fact, AFAIK this is the only way how the 'general case' function is used. I certainly would not expect it to be present in an object file: only (manually or automatically generated from the template) specialized versions go there.

I rarely use manually created specialized cases, and I usually let the compiler generate specialized cases automatically from the template definition when it needs them. The only issue is that this probably leads to a lot of code duplication in the executable, if the same specialized function is added to different object files.

But templates are one of the most advanced topics in C++, and while I have some basic experience with them, there are probably still some things I don't know.

I don't need a specialized function for every type I want to use, just when it's used as a string. All the non-specialized types are automagically generated by the compiler, which is why templates exist in the first place.

I'm not sure if I understand correctly.
You have a header file in which you only declare templates and then you specialise them, right?

From my experience, you can't declare templates without defining them. You must either put a template's definition (without specialisation!) inside a header file (increases compile time), or if you're making a closed source library, you can simply put such a template in a cpp file. I imagine that to use it in other files, you'd have to declare the template somewhere, using the external keyword. Probably in the headers, but I'm not sure if you can even declare a template with the external keyword.

Tell me how you get on.

I have a header file with the class definition, which is included by two cpp files. One contains the implementation and the other uses it, which is the norm. 'extern' is not needed because functions and classes don't need it when you have their prototypes.

 

[Urwumpe]

Nope. That's just a formatting style. The extra '<string>' is also not needed when the compiler can infer the type of the template.

Compilers love it if you use explicit language.

 

[MeDiCS]

Compilers love it if you use explicit language.

:lol:, true. It still gives me the same error thou' :beathead:

 

[Urwumpe]

:lol:, true. It still gives me the same error thou' :beathead:

I'll try it when I am home, I have my holy book of C++ not with me here. (Today is Lua and Windows XP installation day)

 

[Enjo]

I have a header file with the class definition, which is included by two cpp files. One contains the implementation and the other uses it, which is the norm. 'extern' is not needed because functions and classes don't need it when you have their prototypes.

Classes, yes, but not templates, which can't be declared inside classes without the templates' definitions. The idea of using extern goes outside of the class' declaration, if you don't want to define the template inside the class' prototype. Well, I'm not sure about that extern, but I am about that definition inside header as we all suggest, or cpp file. Also my experience tells me that templates can't be just declared as regular class members, because they aren't regular functions.

 

[MeDiCS]

Classes, yes, but not templates, which can't be declared inside classes without the templates' definitions. The idea of using extern goes outside of the class' declaration, if you don't want to define the template inside the class' prototype. Well, I'm not sure about that extern, but I am about that definition inside header as we all suggest, or cpp file. Also my experience tells me that templates can't be just declared as regular class members, because they aren't regular functions.

I was thinking something along those lines: the compiler can't generate the implementations at compile-time because it doesn't know with what types the function will be used. That's only known at link time when the compiler can't do anything about it, and so, it emits an error. This also explains why the only function included in the object file is the specialized method...

Well, gonna restructure my code to include the definition in the header file and test it.

I'll try it when I am home, I have my holy book of C++ not with me here. (Today is Lua and Windows XP installation day)

That'd be excellent! What's the name of the book BTW?

 

[martins]

Erm, well, that's a template class. My problem is regular classes with a template method.

Ah, ok, I missed that. So what about instantiating the function, rather than the whole class, i.e.

template double Test::get_value();
template int Test::get_value();

Incidentally, I notice that your template functions only differ in the return value. The template type doesn't appear in the parameter list. Is that legal? I know in C it wasn't allowed to have functions with the same name that only differed in the return value, because C couldn't distinguish between them. I guess in C++ this may be ok, because the return type is probably included in the mangled function name.

 

[MeDiCS]

Well, it works, but is awfully ugly as I have to define #ifdef guards to avoid redefinition.

This thing is rendering templates mostly useless ¬¬. I'm thinking of writing a 'template-like' function macro to generate get_double_value(), get_uint_value() etc, like C in old times. At least I'll be able to make my code cleaner.
:compbash:

---------- Post added at 03:24 PM ---------- Previous post was at 03:21 PM ----------

Ah, ok, I missed that. So what about instantiating the function, rather than the whole class, i.e.

template double Test::get_value();
template int Test::get_value();

Incidentally, I notice that your template functions only differ in the return value. The template type doesn't appear in the parameter list. Is that legal? I know in C it wasn't allowed to have functions with the same name that only differed in the return value, because C couldn't distinguish between them. I guess in C++ this may be ok, because the return type is probably included in the mangled function name.

Yes, it is legal ONLY if I'm defining the return value as a template, and not as a 'normal' function overloading. But C doesn't support overloading (and that's one of the reasons I like it better. I'm only touching C++ because I'm developing an addon :thumbup.

 

[cjp]

I was thinking something along those lines: the compiler can't generate the implementations at compile-time because it doesn't know with what types the function will be used. That's only known at link time when the compiler can't do anything about it, and so, it emits an error. This also explains why the only function included in the object file is the specialized method...

At link time, all type information is gone! All that's left is a number of object files, and each object file has a number of symbols, which can e.g. be global variables or functions.

So, the linker has nothing to do with how template functions work.

Well, it works, but is awfully ugly as I have to define #ifdef guards to avoid redefinition.

This thing is rendering templates mostly useless ¬¬. I'm thinking of writing a 'template-like' function macro to generate get_double_value(), get_uint_value() etc, like C in old times. At least I'll be able to make my code cleaner.
:compbash:

---------- Post added at 03:24 PM ---------- Previous post was at 03:21 PM ----------


Yes, it is legal ONLY if I'm defining the return value as a template, and not as a 'normal' function overloading. But C doesn't support overloading (and that's one of the reasons I like it better. I'm only touching C++ because I'm developing an addon :thumbup.

If you prefer C over C++, then I can imagine you prefer the approach you describe here. Nobody is stopping you from using C-style programming in C++. Well, at least not for a hobby project. If you're a professional developer, your boss and coworkers might want to stop you.

Having dozens of different functions that essentially do the same thing is not considered clean. You have a lot of code duplication, which makes your program hard to maintain. I still think that C++ templates are the right tool, and if you somehow end up with something that's even uglier than the C approach, then I think it can be done better.

I tried your example, and it works for me (GCC 4.1.1):

template.h:

#include <string>

class Test
{
public:
    template<class T> T get_value ();
};

template <class T> T Test::get_value ()
{
    return T(3.14159);
}

template <> std::string Test::get_value ()
{
    return "pi";
}

test.cpp:

#include <iostream>

using namespace std;

#include "template.h"

int main()
{
    Test testObj;
    cout << testObj.get_value<std::string>() << endl;
    cout << testObj.get_value<int>() << endl;
    cout << testObj.get_value<float>() << endl;

    return 0;
}

output:

pi
3
3.14159

 

[MeDiCS]

At link time, all type information is gone! All that's left is a number of object files, and each object file has a number of symbols, which can e.g. be global variables or functions.

So, the linker has nothing to do with how template functions work.

Yes, it does. Type information is embedded in function symbols (called symbol decoration) so as to support overloading. The problem is that the compiler doesn't generate the right symbols because it doesn't know for which types to generate.

If you prefer C over C++, then I can imagine you prefer the approach you describe here. Nobody is stopping you from using C-style programming in C++. Well, at least not for a hobby project. If you're a professional developer, your boss and coworkers might want to stop you.

Having dozens of different functions that essentially do the same thing is not considered clean. You have a lot of code duplication, which makes your program hard to maintain. I still think that C++ templates are the right tool, and if you somehow end up with something that's even uglier than the C approach, then I think it can be done better.

I wouldn't be implementing a lot of functions, only one function-generator macro and some extra information for the actual function generation:

#define GET_VALUE(type)              \
    type Test::get_##type##_value () \
    {                                \
        //Actual implementation      \
    }

GET_VALUE (double)
GET_VALUE (int)
...

Of course nobody is stopping me from using C-style programming (which, IMO, is sometimes way better than C++). I just want to do the cleanest way, which, in this case, is templates. But if I can't, the second-best alternative is to use function-generator macros.

I tried your example, and it works for me (GCC 4.1.1):[...]

Yes it will work. Unfortunately, the example does not mirror the actual structure: the declaration is on a header file, which gets included by two cpp files, the implementation and another containing a function to use it. That way, it won't work.


Now, the problem is: how do I tell the compiler to generate the right symbols for some types without actually calling the function? Also, I'm curious, if the compiler really does generate non-template symbols, how does STL work (which, AFAIK, is precompiled)?

 

[cjp]

Yes, it does. Type information is embedded in function symbols (called symbol decoration) so as to support overloading. The problem is that the compiler doesn't generate the right symbols because it doesn't know for which types to generate.

OK, admitted, that is true, and if you include debugging information, it's in there too. But that is not used in any way for C++ templates.

I wouldn't be implementing a lot of functions, only one function-generator macro and some extra information for the actual function generation:

#define GET_VALUE(type)        \
    type get_##type##_value () \
    {                          \
        //Actual implementation\
    }

GET_VALUE (double)
GET_VALUE (int)
...

I didn't think of that approach. LOL, that almost looks like a sort of C-style template function!

Now, the problem is: how do I tell the compiler to generate the right symbols for some types without actually calling the function? Also, I'm curious, if the compiler really does generate non-template symbols, how does STL work (which, AFAIK, is precompiled)?

I think this is where you don't understand C++ templates. But let me first give you a new example:

template.h:

#include <string>

class Test
{
public:
    template<class T> T get_value ();
};

template <class T> T Test::get_value ()
{
    return T(3.14159);
}

template <> std::string Test::get_value ();

template.cpp:

#include "template.h"

template <> std::string Test::get_value ()
{
    return "pi";
}

test.cpp:
(identical to the previous example)

As you can see in this example, the specialized function can be placed in a separate source file, and it will also end up in a separate object file, where it will have its own symbol name. I think the same can be done with specific versions of the general function, such as Test::get_value<int> or Test::get_value<float>. However, as far as I understand, you can NOT do the same thing with the template itself!!! You can not have a symbol for "template<class T> T get_value ()" in an object file.

The reason for this is that there is no way for the compiler to generate machine code for a template function. Machine code can only be generated when the exact types of all variables are known, so that the compiler knows how much memory they consume, which processor instructions should be used & so on.

And no, the STL is not precompiled. Maybe there are VCPP "precompiled headers", but there are definitely no object files with machine code for STL. Have a look into the header files of the STL. All the implementation details are there.

 

[MeDiCS]

I think this is where you don't understand C++ templates. But let me first give you a new example:

template.h:

#include <string>

class Test
{
public:
    template<class T> T get_value ();
};

template <class T> T Test::get_value ()
{
    return T(3.14159);
}

template <> std::string Test::get_value ();

template.cpp:

#include "template.h"

template <> std::string Test::get_value ()
{
    return "pi";
}

test.cpp:
(identical to the previous example)

As you can see in this example, the specialized function can be placed in a separate source file, and it will also end up in a separate object file, where it will have its own symbol name. I think the same can be done with specific versions of the general function, such as Test::get_value<int> or Test::get_value<float>. However, as far as I understand, you can NOT do the same thing with the template itself!!! You can not have a symbol for "template<class T> T get_value ()" in an object file.

The reason for this is that there is no way for the compiler to generate machine code for a template function. Machine code can only be generated when the exact types of all variables are known, so that the compiler knows how much memory they consume, which processor instructions should be used & so on.

And no, the STL is not precompiled. Maybe there are VCPP "precompiled headers", but there are definitely no object files with machine code for STL. Have a look into the header files of the STL. All the implementation details are there.

This is actually what I did to make it work, as I've stated a few posts back. The problem is that, because the definition is included in the header, and that the header is copied verbatin by the preprocessor when you use the #include preprocessor command, you'll end up with two or more identical definitions of the same symbol in different object files, which will raise an error by the linker. I've got around that by using #ifdef guards, which makes the code unglier than already is.

But found a workaround to make it compile as I originally wanted by using the nonstandard export keyword: http://www.parashift.com/c++-faq-lite/templates.html#faq-35.14, and http://en.wikipedia.org/wiki/C++#Compatibility, but, unfortunately, is not supported by VC2008 nor GCC :shrug:.

I'll take a look at the STL headers to try to get an insight of how they did it.

---------- Post added at 05:11 PM ---------- Previous post was at 05:08 PM ----------

OK, admitted, that is true, and if you include debugging information, it's in there too. But that is not used in any way for C++ templates.

Unfortunately not. The object file format doesnt support this kind of thing.

I didn't think of that approach. LOL, that almost looks like a sort of C-style template function!

Learned that while reading kernel code :thumbup:.

---------- Post added at 05:25 PM ---------- Previous post was at 05:11 PM ----------

Which STL header should I look? All use of templates up to now are for classes, and not methods...

 

[cjp]

This is actually what I did to make it work, as I've stated a few posts back. The problem is that, because the definition is included in the header, and that the header is copied verbatin by the preprocessor when you use the #include preprocessor command, you'll end up with two or more identical definitions of the same symbol in different object files, which will raise an error by the linker. I've got around that by using #ifdef guards, which makes the code unglier than already is.

I actually tested that code before posting it, and it worked for me. The only thing I can think of now is that I forgot to add the usual #ifdefs to avoid double inclusion:

#ifndef TEMPLATE_H
#define TEMPLATE_H

#include <string>

class Test
{
public:
    template<class T> T get_value ();
};

template <class T> T Test::get_value ()
{
    return T(3.14159);
}

template <> std::string Test::get_value ();

#endif

On which function does the linker give an error? The specialized function for std::string should only be present in the template.o / template.obj file, and the other source files should have it as an external symbol only, because of the declaration in the header file.

I can imagine that, when the general function is used for the same type in different source files (e.g. a.cpp uses Test::get_value<int> and b.cpp also uses Test::get_value<int>), that the same symbol ends up in different object files.

I don't know how this is handled, but I've never had a problem with this (at least not with GCC). I think the compiler should know when a specialized function is generated locally, and it should not export symbols for these. Alternatively, maybe it does export symbols with a special flag / in a special section, so that the C++ linker can remove all the doubles.

---------- Post added at 09:42 PM ---------- Previous post was at 09:31 PM ----------

Which STL header should I look? All use of templates up to now are for classes, and not methods...

Are there no methods inside the classes?

On my system (Ubuntu 9.10 with GCC 4.4.1), the actual implementation of the std::vector class is in bits/stl_vector.h. The header file on my system is copyrighted by Hewlett-Packard Company and Silicon Graphics Computer Systems, Inc., so if yours is not, you probably have a different implementation of the STL.

To give you some fragments from the header file:

[..]
template<typename _Tp, typename _Alloc = std::allocator<_Tp> >
    class vector : protected _Vector_base<_Tp, _Alloc>
    {
[..]
      size_type
      size() const
      { return size_type(this->_M_impl._M_finish - this->_M_impl._M_start);}
[..]
      void
      resize(size_type __new_size, value_type __x = value_type())
      {
    if (__new_size < size())
      _M_erase_at_end(this->_M_impl._M_start + __new_size);
    else
      insert(end(), __new_size - size(), __x);
      }
[..]
    };
[..]

P.S. I believe the above quotation is fair use. Besides, the code is licensed as GPL with additional permissions, so I guess there's no copyright problem anyway.